How is structured output actually enforced?
Two approaches dominate. The older one prompts the model to return JSON and hopes; the model can still emit a stray sentence or a malformed bracket, so a caller ends up writing regex to clean up the mess. The newer, more reliable approach constrains decoding itself: the model only samples tokens that keep the output valid against a JSON Schema, so a malformed response is not possible by construction. This is the difference between asking nicely and enforcing the contract, and it is what makes tool calling dependable enough to wire into production code.
Where does structured output actually matter?
Anywhere a model’s output feeds a machine rather than a person: an AI agent deciding which function to call and with what arguments, a classification step in a pipeline, or a RAG system returning a citation alongside an answer. Free text is for humans; a schema is for the next system in line. Pydantic AI builds an entire agent framework on this idea, validating every model response against typed Python models before application code ever sees it. It does not fix a wrong answer, only a badly shaped one, so accuracy still needs the checks covered in AI agent evals.