Agent returned malformed JSON and broke the next step? Fixing wrong output format

The next step in your pipeline expects clean JSON. The agent gave it JSON wrapped in an explanatory sentence, or with a trailing comma, or cut off halfway through an object. The parser throws, the pipeline breaks, and the failure lands in the downstream code — even though the cause was upstream, in how the agent formatted its output.

Why agents produce malformed output

Two causes account for most of it, and they need different fixes:

• Wrapped or chatty output. The model returns valid JSON but surrounds it with prose ("Here's the result:") or code fences — so a strict parser chokes on the extra text.
• Silent truncation. The output hit the token limit mid-object (finish_reason: "length") and the fragment is invalid JSON. No exception is raised — the SDK hands you the partial content and the parser fails downstream.

The truncation case is the nastier one, because it's silent. The model didn't refuse; it ran out of room. And even finish_reason: "stop" doesn't guarantee valid JSON — the model can end cleanly on malformed output.

The fix

1. Use structured-output / JSON mode with a strict schema so the model is constrained to emit valid, unwrapped JSON.
2. Always validate before passing downstream. Parse and schema-check the output at the boundary; on failure, retry rather than propagating a broken payload.
3. Check finish_reason. If it's "length", the output was truncated — raise max_tokens or paginate at the application layer. Don't ask the model to "continue where it left off"; it reliably fails to produce valid concatenated JSON.
4. Never trust "stop" blindly. Validate every response, even a clean finish.

Diagnose it

Send the trace and Snapback returns the failure — wrong_output_format for wrapped/invalid output, or the LLM-output truncation pattern for a finish_reason: "length" cut-off — with the specific fix for which one you hit:

diagnose_infra_error({error: "finish_reason length incomplete json malformed"})
// → { family: "llm-output",
//      fix: "raise max_tokens / paginate at the app layer; validate every response — don't resume mid-token" }