AI LAB / LLMs / Streaming
Streaming JSON Repair
Exploring whether partial structured output can be validated and repaired during streaming.
LLMsStreamingStructured Output
- EXPERIMENT
- EXP-021
- STATUS
- inconclusive
- DATE
- Jul 14, 2026
- TAGS
- LLMs · Streaming · Structured Output
EXPERIMENT / HYPOTHESIS
Incremental parsing can surface useful validated fields before a complete model response arrives.
- 01Stream 200 nested JSON responses
- 02Parse complete tokens into an incremental tree
- 03Measure time-to-first-valid-field and repair accuracy
- Earlier useful data
- 420ms
- Repair success
- 72%
- Runs
- 200
CONCLUSIONIncremental parsing improved perceived speed, but repair behavior was too provider-specific to standardize without a narrower schema subset.
Approach
The parser emitted only fields whose syntax and local schema constraints were complete, while buffering uncertain branches.
- Incremental tokens
- Field-level validation
- Final full-object check
Why inconclusive
Provider chunk boundaries and partial escaping produced inconsistent repair opportunities. The complexity outweighed the latency benefit for most flows.
- Useful for long outputs
- Risky for side effects
- Needs constrained schemas