Output Format

When a program parses the output, "please return JSON" is a request held together by luck. Pin the format down layer by layer, and watch the parse-failure rate.

Whenever a program has to catch the output, format stops being an aesthetic question and becomes a usability one: free-form text is a liability for the code downstream. Asking for the format in the prompt usually works — and "usually" is not enough, because the share of cases where it fails lands in production as hundreds of parse failures a day.

After reading, you should be able to answer:

• What lies between writing the format down and blocking it during generation?
• Once a format is enforced, what is guaranteed and what is not?
• When one output has to serve both a reader and a program, how do you split it?

A request in the prompt is still only a request

This is the lowest bar to clear: describe the format you want. Two techniques lift the hit rate noticeably — attach a sample of the format (far more effective than describing it, and an application of few-shot examples), and say explicitly "no explanatory text; the first character starts the structure."

But it remains a request, not a guarantee. What it does is probabilistic continuation: the corpus holds far too many answers wrapped in a friendly opener, or fenced inside a code block, so those habits keep a nonzero chance of surfacing. Treat this layer as your only defense and you have accepted a fixed failure rate.

Blocking it during generation is a different kind of measure

Most platforms offer this: hand over a list of fields, and during generation only content matching that list can be produced — an out-of-grammar character never gets emitted at all. That plugs the leak at the source instead of inspecting the result afterwards. Vendor documentation is the authoritative reference for what exactly is enforced. openai-structuredclaude-structured

If it is available, use it — this is the most visible reliability gain there is. But draw its boundary clearly: the structure will be valid; the content may not be. No field is missing, yet a value can be invented; the type lines up, the number does not. Format constraints cannot stop fabrication.

There is a real trade-off to acknowledge: an enforced format narrows the model's freedom. On tasks that need extended reasoning or creative expression, squeezing the output into a strict structure can make the content itself worse. So do not apply the strongest constraint to every call — loose for human-read output, strict for machine-read output.

Whether the format is well shaped decides whether it fills it in right

Constraints alone are not enough. How well the format is shaped directly decides whether the model fills it in correctly. Four lessons.

One, field names should explain themselves. Call a field date when the material contains three different dates, and it can only guess which one you meant; call it signing_date and there is nothing left to guess. It fills values along the literal semantics of the name, so a vague name forces a guess.

Two, do not nest deeply. Past three levels of nesting, the error rate rises noticeably. Flatten whatever can be flattened.

Three, bound your enums. For classification, list every allowed value up front. Skip this and it invents a fresh set of categories — plausibly enough that you may not spot it for a while.

Four, leave a way to say "unsure." This is the one most often forgotten. Make every field required, and the only route left for information the material does not contain is to invent something. Allowing a blank, or accepting a legitimate value like "not in the source," hands it a path to tell the truth — one of the most concrete anti-fabrication moves.

One more point that slips past people: it has to know the format before it starts writing. Laying out the structure first and having it fill the blanks, versus letting it write and then reshaping the result, are two different jobs. The first is filling in; the second is rewriting — and rewriting loses information more often.

Human-read and machine-read outputs collide here, at streaming

Settle this one during design, and plenty of teams only learn it in production.

Streaming does a great deal for perceived speed: characters popping out one at a time, with something visibly happening, beats staring at a spinner. But it conflicts with structured output by nature — a structure that is only half written cannot be parsed. To parse it you must wait for the final chunk, and by then streaming has bought you nothing.

So ask where this output is going, and let the answer decide.

For a person, drop the structure. Turn streaming on and let the result be prose someone can read, rather than bolting a structure on to make the screen look tidy.

For a program, drop the streaming. Data extraction, labelling, classification — the output exists for code to consume, the user never sees the intermediate state, and streaming earns nothing. Turn it off and use strict constraints.

For both, split it in two. This is the most realistic case — you want an answer that reads naturally on screen, and you also need structured metadata. The approach: put the human-facing body first and append a short structured block at the end. The frontend renders the body; the program parses the tail. Or go further and make two calls, each doing one job.

This one gets decided during design, not after the frontend is built and the formats will not line up. "Is this output for a person or for a program?" is a question to answer before you write the prompt.

You write the validation layer, and you watch its number

Even with the earlier layers in place, this one stays mandatory. How to handle a parse failure, a missing field, a value that is plainly unreasonable — those branches all have to be written.

Three common fallbacks: retry once (pass the error back and let it correct itself), degrade (switch models or fall back to a simpler mode), refuse (tell the user plainly that this attempt failed, instead of putting garbage in front of them).

Once the branches exist, something has to watch them: make the parse-failure rate a metric. Do not wait for user complaints — watch the number yourself. It earns its place three ways.

One, it tells you whether layer one is enough. If you only ask in the prompt, the failure rate is your evidence for whether you need layer two.

Two, when it jumps, something upstream changed. Especially if you use a model name that tracks the provider's latest — a version swap changes format-following habits, and your failure rate is the earliest signal. The right response is to re-run your samples and scope the impact, not to patch in compatibility logic on the spot.

Three, it is the only format problem you can catch automatically. Wrong content needs human eyes or evaluation; a broken structure is something a machine detects instantly — and that signal should not be left sitting in a log.

One discipline: do not suppress this number with compatibility code. Stuffing more tolerance branches into the parser looks like a lower failure rate, but it converts a visible failure into a silently wrong value — which is far worse. Either fix the format at the source or refuse the output outright.

With that, the format is pinned down. One variable remains unavoidable: it samples differently each run, so the same input does not guarantee the same output. Treating that randomness as a dial you control is what Randomness is designed in, not a bug covers.