Structured Output vs Free-Form Text for Tool-Calling LLMs

Tool-calling LLMs can return either free-form text or schema-constrained structured output. The right choice depends on the consumer, not on the model.

Use structured output when

• Another piece of code will parse the response.
• You need a closed set of values (a status, a code, a category).
• You want to detect refusal or empty answers with a deterministic check.

Use free-form text when

• A human will read it.
• The response is an explanation, a summary, or a list of reasons.
• The shape is genuinely unknown ahead of time.

Failure modes of forced structure

• The model invents fields the schema does not forbid. Tighten the schema.
• The model returns null for fields it does not know. Make those fields optional and add validation at the consumer.
• The model refuses to answer because the schema does not match its plan. Loosen the schema or split the call.

Failure modes of free form

• The model wraps the answer in markdown fences. Strip them at the consumer.
• The model answers in a different language. Detect and either translate or re-prompt.

The rule of thumb: parse it later, structure it now. Show it to a human, let it be prose.