Structured output is a large language model's response constrained to a defined schema, usually JSON, so a program can parse the result directly instead of pattern matching free text, making the model's response reliable enough to feed straight into another system or tool call.
How is structured output actually enforced?
Two approaches dominate. The older one prompts the model to return JSON and hopes; the model can still emit a stray sentence or a malformed bracket, so a caller ends up writing regex to clean up the mess. The newer, more reliable approach constrains decoding itself: the model only samples tokens that keep the output valid against a JSON Schema, so a malformed response is not possible by construction. This is the difference between asking nicely and enforcing the contract, and it is what makes tool calling dependable enough to wire into production code.
Where does structured output actually matter?
Anywhere a model’s output feeds a machine rather than a person: an AI agent deciding which function to call and with what arguments, a classification step in a pipeline, or a RAG system returning a citation alongside an answer. Free text is for humans; a schema is for the next system in line. It does not fix a wrong answer, only a badly shaped one, so accuracy still needs the checks covered in AI agent evals.
Frequently asked questions
Does structured output guarantee a correct answer?
No. It guarantees a well-formed answer that matches the schema, not a factually correct one. A malformed response and a wrong-but-valid response are different failure modes, and structured output only prevents the first.
Is structured output the same as function calling?
They overlap. Function calling is a specific use of structured output where the schema describes a tool's arguments; structured output is the broader mechanism, used for classification, extraction, and any response a program needs to parse.