AI LAB / LLMs / Structured Output
Can LLMs Generate Schemas?
Measuring structured schema generation across models, prompts, and repair strategies.
LLMsStructured OutputEvaluation
- EXPERIMENT
- EXP-016
- STATUS
- complete
- DATE
- Aug 18, 2026
- TAGS
- LLMs · Structured Output · Evaluation
EXPERIMENT / HYPOTHESIS
Schema-constrained generation plus one validation-aware repair will exceed 95% syntactic validity.
- 01Generate 80 schemas from product descriptions
- 02Validate syntax and semantic constraints
- 03Retry invalid output once with structured errors
- First-pass valid
- 89.8%
- After repair
- 97.5%
- Runs
- 80
CONCLUSIONA single targeted repair recovered most syntax failures, but semantic field mistakes still required domain validation.
Test design
Prompts ranged from simple forms to nested workflow contracts with discriminated variants and cross-field constraints.
- Syntax validity
- Constraint coverage
- Unnecessary fields
Finding
Provider schema modes solved formatting but not product semantics. Deterministic domain checks remained necessary.
- Validate twice
- Repair narrowly
- Do not trust plausible fields