Why JSON breaks LLM streaming
Strict JSON output looks clean on paper, but token-by-token delivery turns it fragile in production.
- LLM
- Streaming
- Reliability
Core Failure Mode
JSON assumes completeness. Streaming is incomplete by definition.
When we parse too early, we fail. When we wait for completion, we lose responsiveness.
Better Pattern
Stream into a tolerant intermediate format, then transform into validated JSON chunks that can be safely applied.
Takeaway
For builder tools, perceived reliability beats strict formatting purity.