Migrating from LangChain OutputFixingParser

OutputFixingParser works but has two problems: latency and retries. Shim fixes both.

The Problems

1. Latency

OutputFixingParser waits for the full LLM output before attempting repair. This adds 2-5 seconds of delay.

# OutputFixingParser flow:
# 1. Wait for full LLM output (2-5s)
# 2. Attempt parse
# 3. If failed, send to repair LLM (another 2-5s)
# Total: 4-10s latency

2. Retries are Expensive

OutputFixingParser sends broken JSON to another LLM call:

parser = OutputFixingParser.from_llm(
    parser=JsonOutputParser(),
    llm=ChatOpenAI(model="gpt-4")
)

# If parsing fails:
# - Sends broken JSON to GPT-4
# - GPT-4 attempts to fix it
# - Costs $0.03/1K tokens (GPT-4 input)
# - Takes 2-5s

Cost: 1,000 failed parses/day = $30/day in retry costs.

Migration Path

Before: LangChain

from langchain.output_parsers import JsonOutputParser
from langchain_openai import ChatOpenAI
from langchain.output_parsers import OutputFixingParser

# Create parser
parser = JsonOutputParser()

# Wrap with fixing parser
fixing_parser = OutputFixingParser.from_llm(
    parser=parser,
    llm=ChatOpenAI(model="gpt-4")
)

# Invoke chain
chain = prompt | llm | fixing_parser
result = chain.invoke({"query": "user input"})

After: Shim

import requests
import os

# Get LLM output
chain = prompt | llm
raw_output = chain.invoke({"query": "user input"})

# Repair with Shim
response = requests.post(
    'https://api.shim.so/v1/repair',
    headers={'Authorization': f'Bearer {os.environ["SHIM_API_KEY"]}'},
    json={'raw_output': raw_output}
)

result = response.json()

if result['success']:
    data = result['repaired']
    print(f"Confidence: {result['metadata']['confidence']}")
else:
    print(f"Repair failed: {result['metadata']['errors']}")

Benefits

Metric	OutputFixingParser	Shim
Latency	4-10s	`<10ms`
Retry cost	$0.03/repair	$0.001/repair
Streaming	No	Yes
Schema validation	No	Yes
Confidence scoring	No	Yes

With Schema Validation

Before: LangChain (Pydantic)

from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field

class Person(BaseModel):
    name: str = Field(description="Person's name")
    age: int = Field(description="Person's age")

parser = PydanticOutputParser(pydantic_object=Person)

fixing_parser = OutputFixingParser.from_llm(
    parser=parser,
    llm=ChatOpenAI(model="gpt-4")
)

chain = prompt | llm | fixing_parser
result = chain.invoke({"query": "user input"})

After: Shim (JSON Schema)

import requests
import os

# Convert Pydantic to JSON Schema
schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "number"}
    },
    "required": ["name", "age"]
}

# Get LLM output
raw_output = chain.invoke({"query": "user input"})

# Repair with schema
response = requests.post(
    'https://api.shim.so/v1/repair',
    headers={'Authorization': f'Bearer {os.environ["SHIM_API_KEY"]}'},
    json={
        'raw_output': raw_output,
        'schema': schema,
        'mode': 'strict'
    }
)

result = response.json()

if result['success']:
    person = result['repaired']
    print(f"Name: {person['name']}, Age: {person['age']}")

Streaming Support

Before: No Streaming

LangChain parsers don’t support streaming. Must wait for full output.

After: Shim Streaming

import requests
import os

api_key = os.environ["SHIM_API_KEY"]
headers = {'Authorization': f'Bearer {api_key}'}

# Start session
session_resp = requests.post(
    'https://api.shim.so/v1/repair/stream/start',
    headers=headers,
    json={'schema': schema, 'mode': 'strict'}
)
session_id = session_resp.json()['session_id']

# Stream LLM chunks
for chunk in llm.stream(prompt):
    # Push to Shim
    push_resp = requests.post(
        'https://api.shim.so/v1/repair/stream/push',
        headers=headers,
        json={'session_id': session_id, 'chunk': chunk}
    )

    state = push_resp.json()['state']

    # Show preview if parseable
    if state['safe_to_emit'] and state['partial']:
        print(f"Preview: {state['partial']}")

# Finalize
final_resp = requests.post(
    'https://api.shim.so/v1/repair/stream/finalize',
    headers=headers,
    json={'session_id': session_id}
)

result = final_resp.json()['repaired']

Using the TypeScript SDK

For Node.js/TypeScript projects:

import { ShimClient } from 'shim-sdk';

const shim = new ShimClient({ apiKey: process.env.SHIM_API_KEY });

// Batch repair
const result = await shim.repair({
  raw_output: llmOutput,
  schema: {
    type: 'object',
    properties: {
      name: { type: 'string' },
      age: { type: 'number' }
    },
    required: ['name', 'age']
  },
  mode: 'strict'
});

if (result.success) {
  console.log('Repaired:', result.repaired);
  console.log('Confidence:', result.metadata.confidence);
}

Cost Comparison

OutputFixingParser

Scenario: 10,000 repairs/month, 10% failure rate

Repair LLM calls: 10,000 × 10% = 1,000 calls
Average tokens per repair: 500 tokens
GPT-4 input cost: $0.03/1K tokens

Monthly cost: 1,000 × (500/1000) × $0.03 = $15/month

Shim Pro Tier

Scenario: 10,000 repairs/month

Base cost: $29/month (includes 100K repairs)
Overage: $0 (well under limit)

Monthly cost: $29/month

Savings: OutputFixingParser is cheaper at low volume, but Shim is faster and more reliable. Break-even: ~50K repairs/month.

Confidence Scoring

Shim adds confidence levels that LangChain doesn’t provide:

result = shim.repair({'raw_output': llm_output})

if result['success']:
    confidence = result['metadata']['confidence']

    if confidence == 'high':
        # Safe to use
        return result['repaired']
    elif confidence == 'medium':
        # Log for review
        logger.info('Medium confidence repair', result['metadata'])
        return result['repaired']
    elif confidence == 'low':
        # Alert for manual review
        alert_ops('Low confidence repair')
        return result['repaired']

Error Handling

Before: Exceptions

try:
    result = fixing_parser.invoke(llm_output)
except Exception as e:
    logger.error(f'Parser failed: {e}')
    return None

After: Structured Errors

result = shim.repair({'raw_output': llm_output})

if not result['success']:
    errors = result['metadata']['errors']

    for error in errors:
        logger.error(f"Repair failed: {error['code']}", {
            'message': error['message'],
            'recoverable': error['recoverable']
        })

    # Check if recoverable
    if all(e['recoverable'] for e in errors):
        # Retry logic
        return retry_repair(llm_output)

    return None

Migration Checklist

Sign up at console.shim.so/signup
Get API key from console
Install shim-sdk (if using TypeScript)
Replace OutputFixingParser with Shim API call
Add schema validation (optional)
Implement confidence-based handling
Add structured error handling
Test with production traffic
Remove LangChain parser dependencies

Hybrid Approach

Keep LangChain, add Shim as a repair layer:

from langchain.output_parsers import JsonOutputParser
import requests
import os

parser = JsonOutputParser()

def parse_with_shim(llm_output: str):
    # Try LangChain parser first
    try:
        return parser.parse(llm_output)
    except Exception:
        # Fall back to Shim
        response = requests.post(
            'https://api.shim.so/v1/repair',
            headers={'Authorization': f'Bearer {os.environ["SHIM_API_KEY"]}'},
            json={'raw_output': llm_output}
        )

        result = response.json()

        if result['success']:
            return result['repaired']

        raise ValueError('Both parsers failed')

# Use in chain
chain = prompt | llm | parse_with_shim

FAQ

Can I use Shim with LangChain?

Yes. Use Shim as a post-processing step after LLM output.

Do I need to remove LangChain?

No. Shim complements LangChain. Replace only the parser.

What about LCEL (LangChain Expression Language)?

Shim works with LCEL. Add it as a final step in the chain.

Does Shim support Pydantic?

Shim uses JSON Schema. Convert Pydantic models with model.model_json_schema().

Can I use OutputFixingParser as a fallback?

Yes, but Shim is faster and cheaper. Fallback not needed.

Next Steps

Quick Start

Get your first repair working

Schema Validation

Add JSON Schema validation

TypeScript SDK

Use the official SDK

Confidence Levels

Understand confidence scoring

​Migrating from LangChain OutputFixingParser

​The Problems

​1. Latency

​2. Retries are Expensive

​Migration Path

​Before: LangChain

​After: Shim

​Benefits

​With Schema Validation

​Before: LangChain (Pydantic)

​After: Shim (JSON Schema)

​Streaming Support

​Before: No Streaming

​After: Shim Streaming

​Using the TypeScript SDK

​Cost Comparison

​OutputFixingParser

​Shim Pro Tier

​Confidence Scoring

​Error Handling

​Before: Exceptions

​After: Structured Errors

​Migration Checklist

​Hybrid Approach

​FAQ

​Can I use Shim with LangChain?

​Do I need to remove LangChain?

​What about LCEL (LangChain Expression Language)?

​Does Shim support Pydantic?

​Can I use OutputFixingParser as a fallback?

​Next Steps