> ## Documentation Index > Fetch the complete documentation index at: https://docs.shim.so/llms.txt > Use this file to discover all available pages before exploring further. # Confidence Levels > How Shim scores repair safety # Confidence Levels Every repair response includes a confidence score: `high`, `medium`, or `low`. *** ## The Three Levels | Level | Meaning | Repairs Applied | Action | | ---------- | --------------------------------------- | ---------------------------------- | --------------- | | **high** | Structural fixes only, or perfect input | Brackets, commas, fences (or none) | Use immediately | | **medium** | Schema repairs or warnings | Type coercion, field validation | Review in logs | | **low** | Ambiguous repairs | Multiple interpretations | Manual review | *** ## How Confidence is Calculated Shim uses **monotonic pessimistic scoring**: any low-confidence repair downgrades the entire score. ### Scoring Rules ```typescript theme={null} function calculateConfidence(syntaxRepairs, schemaRepairs, warnings): ConfidenceLevel { // Any low-confidence repair → low if ([...syntaxRepairs, ...schemaRepairs].some(r => r.confidence === 'low')) { return 'low'; } // Any warnings → medium (at best) if (warnings.length > 0) { return 'medium'; } // Any schema repairs → medium (at best) if (schemaRepairs.length > 0) { return 'medium'; } // Only syntax repairs or no repairs → high return 'high'; } ``` ### Why Pessimistic? **Contract:** Confidence never overstates safety. `high` means truly safe. `low` means truly risky. *** ## High Confidence ### Repairs Applied * Closing brackets: `{"name": "John"` → `{"name": "John"}` * Trailing commas: `{"a": 1, "b": 2,}` → `{"a": 1, "b": 2}` * Markdown fences: ` ```json {"a": 1} ``` ` → `{"a": 1}` * Smart quotes: `{"name": "John"}` → `{"name": "John"}` * Leading/trailing text: `Here's the data: {"a": 1}` → `{"a": 1}` ### Why High? These are **unambiguous structural fixes**. No semantic changes. No data loss. ### Example Response ```json theme={null} { "success": true, "repaired": { "name": "John", "age": 30 }, "metadata": { "confidence": "high", "syntax_repairs": [ { "type": "closed_object_bracket", "confidence": "high", "safe": true } ], "schema_repairs": [], "warnings": [] } } ``` ### What to Do Use immediately. No review needed. ```typescript theme={null} const result = await shim.repair({ raw_output: input }); if (result.success && result.metadata.confidence === 'high') { return result.repaired; // Safe } ``` *** ## Medium Confidence ### Repairs Applied * Type coercion: `"30"` → `30` * Unquoted keys: `{name: "test"}` → `{"name": "test"}` * Unquoted values: `{status: active}` → `{status: "active"}` * JavaScript values: `NaN` → `null`, `Infinity` → `null` * Single quotes: `{'key': 'value'}` → `{"key": "value"}` ### Why Medium? These are **policy decisions**. Safe, but not unambiguous. Example: `"30"` could be a string or number. Shim coerces based on schema. ### Example Response ```json theme={null} { "success": true, "repaired": { "name": "John", "age": 30 }, "metadata": { "confidence": "medium", "syntax_repairs": [], "schema_repairs": [ { "type": "type_coercion", "confidence": "high", "safe": true, "field": "age", "original_value": "30", "repaired_value": 30 } ], "warnings": [] } } ``` ### What to Do Use immediately, but log for review. ```typescript theme={null} const result = await shim.repair({ raw_output: input }); if (result.success && result.metadata.confidence === 'medium') { logger.info('Medium confidence repair', { repairs: result.metadata.schema_repairs }); return result.repaired; } ``` *** ## Low Confidence ### Repairs Applied * Ambiguous type inference (no schema provided) * Multiple repair interpretations possible * Complex nested structure repairs ### Why Low? Shim had to **guess** at the correct fix. Multiple interpretations were possible. ### Example Response ```json theme={null} { "success": true, "repaired": { "name": "John", "status": "active" }, "metadata": { "confidence": "low", "syntax_repairs": [], "schema_repairs": [ { "type": "inferred_unquoted_value", "confidence": "low", "safe": false, "field": "status", "original_value": "active", "repaired_value": "active" } ], "warnings": [ { "code": "AMBIGUOUS_REPAIR", "message": "Unquoted value could be identifier or string", "severity": "medium" } ] } } ``` ### What to Do Alert for manual review. ```typescript theme={null} const result = await shim.repair({ raw_output: input }); if (result.success && result.metadata.confidence === 'low') { logger.warn('Low confidence repair', { repairs: result.metadata.repairs_applied, warnings: result.metadata.warnings }); // Alert ops team alertOps('Low confidence Shim repair', { input_preview: input.substring(0, 100) }); return result.repaired; } ``` *** ## Perfect Input (High Confidence) ### Meaning Input was already valid JSON. No repairs needed. Confidence is still `high`. ### Example Response ```json theme={null} { "success": true, "repaired": { "name": "John", "age": 30 }, "metadata": { "confidence": "high", "was_repaired": false, "syntax_repairs": [], "schema_repairs": [], "warnings": [] } } ``` ### What to Do Use immediately. Check `was_repaired: false` to distinguish perfect input from repaired input. ```typescript theme={null} const result = await shim.repair({ raw_output: input }); if (result.success && !result.metadata.was_repaired) { // Perfect input, no repairs needed return result.repaired; } ``` *** ## Confidence by Repair Type | Repair Type | Confidence | Safe? | | ----------------------------- | ---------- | ----- | | **Syntax Repairs** | | | | Closed brackets | High | Yes | | Trailing commas | High | Yes | | Markdown fences | High | Yes | | Leading/trailing text | High | Yes | | Smart quotes | High | Yes | | Multiple commas | High | Yes | | Malformed decimals | High | Yes | | Incomplete numbers | High | Yes | | **Schema Repairs** | | | | Type coercion (with schema) | High | Yes | | Unquoted keys | High | Yes | | Single quotes | High | Yes | | JavaScript comments | High | Yes | | JavaScript values (NaN) | High | Yes | | Unquoted values (with schema) | Medium | Yes | | Extra field removal | Medium | Yes | | **Ambiguous Repairs** | | | | Type inference (no schema) | Low | No | | Unquoted value inference | Low | No | | Multiple interpretations | Low | No | *** ## Strict vs Lenient Mode Mode affects warnings, not confidence: ### Strict Mode (default) High-severity warnings become errors. Repair fails. ```typescript theme={null} const result = await shim.repair({ raw_output: input, schema, mode: 'strict' }); // If high-severity warning: // success: false // errors: [{ code: 'SCHEMA_VALIDATION_FAILED' }] ``` ### Lenient Mode All warnings allowed. Repair succeeds. ```typescript theme={null} const result = await shim.repair({ raw_output: input, schema, mode: 'lenient' }); // Even with high-severity warning: // success: true // confidence: 'medium' or 'low' // warnings: [{ severity: 'high' }] ``` *** ## Best Practices ### 1. Always Provide Schema Schema enables high-confidence type coercion: ```typescript theme={null} // Without schema: low confidence const result = await shim.repair({ raw_output: '{"age": "30"}' }); // confidence: 'low' (type unclear) // With schema: medium confidence (schema repair applied) const result = await shim.repair({ raw_output: '{"age": "30"}', schema: { type: 'object', properties: { age: { type: 'number' } } } }); // confidence: 'medium' (type_coercion is a schema repair) ``` ### 2. Log Medium/Low Repairs Track repair patterns over time: ```typescript theme={null} if (['medium', 'low'].includes(result.metadata.confidence)) { logger.info('Non-high confidence repair', { confidence: result.metadata.confidence, repairs: result.metadata.repairs_applied }); } ``` ### 3. Alert on Low Confidence Spikes ```typescript theme={null} const lowConfidenceRate = await metrics.query( 'shim.confidence.low', '5m' ); if (lowConfidenceRate > 0.1) { alertOps('High rate of low-confidence repairs'); } ``` ### 4. Review Repair Logs Check dashboard for confidence distribution: ```bash theme={null} # Expected distribution: # high: 75-90% (includes perfect input + structural-only repairs) # medium: 10-20% # low: <5% ``` *** ## Next Steps Full response schema Handle low confidence repairs Improve confidence with schemas Debug confidence issues