Confidence Levels

Every repair response includes a confidence score: high, medium, or low.

The Three Levels

Level	Meaning	Repairs Applied	Action
high	Structural fixes only, or perfect input	Brackets, commas, fences (or none)	Use immediately
medium	Schema repairs or warnings	Type coercion, field validation	Review in logs
low	Ambiguous repairs	Multiple interpretations	Manual review

How Confidence is Calculated

Shim uses monotonic pessimistic scoring: any low-confidence repair downgrades the entire score.

Scoring Rules

function calculateConfidence(syntaxRepairs, schemaRepairs, warnings): ConfidenceLevel {
  // Any low-confidence repair → low
  if ([...syntaxRepairs, ...schemaRepairs].some(r => r.confidence === 'low')) {
    return 'low';
  }

  // Any warnings → medium (at best)
  if (warnings.length > 0) {
    return 'medium';
  }

  // Any schema repairs → medium (at best)
  if (schemaRepairs.length > 0) {
    return 'medium';
  }

  // Only syntax repairs or no repairs → high
  return 'high';
}

Why Pessimistic?

Contract: Confidence never overstates safety. high means truly safe. low means truly risky.

High Confidence

Repairs Applied

Closing brackets: {"name": "John" → {"name": "John"}
Trailing commas: {"a": 1, "b": 2,} → {"a": 1, "b": 2}
Markdown fences: ```json {"a": 1} ``` → {"a": 1}
Smart quotes: {"name": "John"} → {"name": "John"}
Leading/trailing text: Here's the data: {"a": 1} → {"a": 1}

Why High?

These are unambiguous structural fixes. No semantic changes. No data loss.

Example Response

{
  "success": true,
  "repaired": { "name": "John", "age": 30 },
  "metadata": {
    "confidence": "high",
    "syntax_repairs": [
      {
        "type": "closed_object_bracket",
        "confidence": "high",
        "safe": true
      }
    ],
    "schema_repairs": [],
    "warnings": []
  }
}

What to Do

Use immediately. No review needed.

const result = await shim.repair({ raw_output: input });

if (result.success && result.metadata.confidence === 'high') {
  return result.repaired; // Safe
}

Medium Confidence

Repairs Applied

Type coercion: "30" → 30
Unquoted keys: {name: "test"} → {"name": "test"}
Unquoted values: {status: active} → {status: "active"}
JavaScript values: NaN → null, Infinity → null
Single quotes: {'key': 'value'} → {"key": "value"}

Why Medium?

These are policy decisions. Safe, but not unambiguous. Example: "30" could be a string or number. Shim coerces based on schema.

Example Response

{
  "success": true,
  "repaired": { "name": "John", "age": 30 },
  "metadata": {
    "confidence": "medium",
    "syntax_repairs": [],
    "schema_repairs": [
      {
        "type": "type_coercion",
        "confidence": "high",
        "safe": true,
        "field": "age",
        "original_value": "30",
        "repaired_value": 30
      }
    ],
    "warnings": []
  }
}

What to Do

Use immediately, but log for review.

const result = await shim.repair({ raw_output: input });

if (result.success && result.metadata.confidence === 'medium') {
  logger.info('Medium confidence repair', {
    repairs: result.metadata.schema_repairs
  });
  return result.repaired;
}

Low Confidence

Repairs Applied

Ambiguous type inference (no schema provided)
Multiple repair interpretations possible
Complex nested structure repairs

Why Low?

Shim had to guess at the correct fix. Multiple interpretations were possible.

Example Response

{
  "success": true,
  "repaired": { "name": "John", "status": "active" },
  "metadata": {
    "confidence": "low",
    "syntax_repairs": [],
    "schema_repairs": [
      {
        "type": "inferred_unquoted_value",
        "confidence": "low",
        "safe": false,
        "field": "status",
        "original_value": "active",
        "repaired_value": "active"
      }
    ],
    "warnings": [
      {
        "code": "AMBIGUOUS_REPAIR",
        "message": "Unquoted value could be identifier or string",
        "severity": "medium"
      }
    ]
  }
}

What to Do

Alert for manual review.

const result = await shim.repair({ raw_output: input });

if (result.success && result.metadata.confidence === 'low') {
  logger.warn('Low confidence repair', {
    repairs: result.metadata.repairs_applied,
    warnings: result.metadata.warnings
  });

  // Alert ops team
  alertOps('Low confidence Shim repair', {
    input_preview: input.substring(0, 100)
  });

  return result.repaired;
}

Perfect Input (High Confidence)

Meaning

Input was already valid JSON. No repairs needed. Confidence is still high.

Example Response

{
  "success": true,
  "repaired": { "name": "John", "age": 30 },
  "metadata": {
    "confidence": "high",
    "was_repaired": false,
    "syntax_repairs": [],
    "schema_repairs": [],
    "warnings": []
  }
}

What to Do

Use immediately. Check was_repaired: false to distinguish perfect input from repaired input.

const result = await shim.repair({ raw_output: input });

if (result.success && !result.metadata.was_repaired) {
  // Perfect input, no repairs needed
  return result.repaired;
}

Confidence by Repair Type

Repair Type	Confidence	Safe?
Syntax Repairs
Closed brackets	High	Yes
Trailing commas	High	Yes
Markdown fences	High	Yes
Leading/trailing text	High	Yes
Smart quotes	High	Yes
Multiple commas	High	Yes
Malformed decimals	High	Yes
Incomplete numbers	High	Yes
Schema Repairs
Type coercion (with schema)	High	Yes
Unquoted keys	High	Yes
Single quotes	High	Yes
JavaScript comments	High	Yes
JavaScript values (NaN)	High	Yes
Unquoted values (with schema)	Medium	Yes
Extra field removal	Medium	Yes
Ambiguous Repairs
Type inference (no schema)	Low	No
Unquoted value inference	Low	No
Multiple interpretations	Low	No

Strict vs Lenient Mode

Mode affects warnings, not confidence:

Strict Mode (default)

High-severity warnings become errors. Repair fails.

const result = await shim.repair({
  raw_output: input,
  schema,
  mode: 'strict'
});

// If high-severity warning:
// success: false
// errors: [{ code: 'SCHEMA_VALIDATION_FAILED' }]

Lenient Mode

All warnings allowed. Repair succeeds.

const result = await shim.repair({
  raw_output: input,
  schema,
  mode: 'lenient'
});

// Even with high-severity warning:
// success: true
// confidence: 'medium' or 'low'
// warnings: [{ severity: 'high' }]

Best Practices

1. Always Provide Schema

Schema enables high-confidence type coercion:

// Without schema: low confidence
const result = await shim.repair({
  raw_output: '{"age": "30"}'
});
// confidence: 'low' (type unclear)

// With schema: medium confidence (schema repair applied)
const result = await shim.repair({
  raw_output: '{"age": "30"}',
  schema: {
    type: 'object',
    properties: { age: { type: 'number' } }
  }
});
// confidence: 'medium' (type_coercion is a schema repair)

2. Log Medium/Low Repairs

Track repair patterns over time:

if (['medium', 'low'].includes(result.metadata.confidence)) {
  logger.info('Non-high confidence repair', {
    confidence: result.metadata.confidence,
    repairs: result.metadata.repairs_applied
  });
}

3. Alert on Low Confidence Spikes

const lowConfidenceRate = await metrics.query(
  'shim.confidence.low',
  '5m'
);

if (lowConfidenceRate > 0.1) {
  alertOps('High rate of low-confidence repairs');
}

4. Review Repair Logs

Check dashboard for confidence distribution:

# Expected distribution:
# high: 75-90%  (includes perfect input + structural-only repairs)
# medium: 10-20%
# low: <5%

Next Steps

Response Object

Full response schema

Error Handling

Handle low confidence repairs

Schema Validation

Improve confidence with schemas

Troubleshooting

Debug confidence issues

​Confidence Levels

​The Three Levels

​How Confidence is Calculated

​Scoring Rules

​Why Pessimistic?

​High Confidence

​Repairs Applied

​Why High?

​Example Response

​What to Do

​Medium Confidence

​Repairs Applied

​Why Medium?

​Example Response

​What to Do

​Low Confidence

​Repairs Applied

​Why Low?

​Example Response

​What to Do

​Perfect Input (High Confidence)

​Meaning

​Example Response

​What to Do

​Confidence by Repair Type

​Strict vs Lenient Mode

​Strict Mode (default)

​Lenient Mode

​Best Practices

​1. Always Provide Schema

​2. Log Medium/Low Repairs

​3. Alert on Low Confidence Spikes

​4. Review Repair Logs

​Next Steps

Response Object

Error Handling

Schema Validation

Troubleshooting

Confidence Levels

The Three Levels

How Confidence is Calculated

Scoring Rules

Why Pessimistic?

High Confidence

Repairs Applied

Why High?

Example Response

What to Do

Medium Confidence

Repairs Applied

Why Medium?

Example Response

What to Do

Low Confidence

Repairs Applied

Why Low?

Example Response

What to Do

Perfect Input (High Confidence)

Meaning

Example Response

What to Do

Confidence by Repair Type

Strict vs Lenient Mode

Strict Mode (default)

Lenient Mode

Best Practices

1. Always Provide Schema

2. Log Medium/Low Repairs

3. Alert on Low Confidence Spikes

4. Review Repair Logs

Next Steps