Aiodic

Back to the table

39 Intermediate So Structured Output Prompting

Structured Output

Getting answers in a fixed format like JSON.

What it is

Structured output means getting an AI's answer back in a fixed, predictable shape instead of as free-flowing conversation. Rather than "Sure! Here's what I found in that email…", you get the same handful of labelled fields every single time, in the same order, with the same names, ready to drop into a spreadsheet, a database, or another program.

The easiest way to picture it is a paper form. Imagine a customer enquiry form with boxes for Name, Email, Product, and Question. Whoever fills it in, the boxes are always in the same place, so the person processing the forms knows exactly where to look. Free text is like a customer writing you a letter: the same information might be in there, but it could be anywhere, phrased any way, or missing altogether. Structured output asks the AI to fill in the form rather than write the letter.

The most common format for these "forms" is JSON, short for JavaScript Object Notation. Despite the name, you don't need to know any programming to read it. It's just labels and values, wrapped in curly brackets. A filled-in enquiry form in JSON looks like this:

{
  "name": "Dana Ruiz",
  "email": "[email protected]",
  "product": "Watercolour starter kit",
  "question": "Does it ship to Canada?",
  "urgent": false
}

Each line is one box on the form: the label on the left in quotation marks, the answer on the right. Text goes in quotes, numbers don't, and yes-or-no answers are written as true or false. Square brackets hold a list, such as several tags. That's nearly all there is to it. Almost every piece of business software can read JSON, which is why it has become the standard way for AI to hand information to other programs. Other structured formats exist too: CSV (comma-separated values, the plain-text format every spreadsheet can open), tables, and the tagged style described in XML Tags.

The blank form itself, before anyone fills it in, has a name too: a schema. A schema is a description of what the answer must look like. It lists every field, says what kind of value each one holds (text, a number, a date, true or false, one of a fixed set of choices), and says which fields are required. "There must be an email field, it must be text, and the category must be one of Sales, Support, or Complaint" is a schema written in plain English. Programmers write the same thing in a standard called JSON Schema, but the idea is identical.

There are two very different ways to get structured output, and the difference is the most important thing in this article:

The first approach is a request. The second is a guarantee about the shape, though not about the truth of what's inside, a distinction we'll come back to.

Why it matters

It's the bridge between AI and everything else you use. A chatbot answer is for a human to read. Most of the useful work in a business happens in spreadsheets, email tools, online shops, booking systems, and automation services like Zapier or Make. None of those can read "Sure, here you go!". All of them can read a field called email. Structured output is what lets an AI step sit in the middle of an automated process, which is the basis of workflow automation.

It turns one-off help into repeatable processing. Asking an AI to summarise one customer email is handy. Asking it to turn 300 emails into 300 identical spreadsheet rows, each with name, order number, problem type and urgency, is a different kind of value altogether. It only works if row 147 has exactly the same shape as row 1.

It makes answers checkable. Once an answer arrives in fields, ordinary software can test it before anyone relies on it: is the date a real date, is the price a number, is the category one of the allowed choices? This is one of the most dependable guardrails there is, and it also makes evals far easier, because a computer can mark a field right or wrong in an instant.

It forces you to be clear about what you want. Writing a schema makes you decide, up front, exactly which facts you need and what counts as a valid answer. Many disappointing AI results come from vague requests. A schema is the opposite of vague.

It's how tools and agents work behind the scenes. When an AI assistant books a meeting or looks something up, it has to ask the software for that action in a precise format: which tool, which date, which search terms. That's function calling, and it's structured output applied to actions rather than answers. Much of what makes an AI agent reliable is that its requests arrive in a shape the software can understand.

For a creator or small business, typical uses include:

Real-world examples

OpenAI: from "valid JSON" to "matches your schema" (2023 and 2024)

On 6 November 2023, at its first developer conference, DevDay, OpenAI introduced what it called JSON mode. Developers could switch on a setting that made the model produce syntactically valid JSON, meaning the brackets and quotes would always be in order. It was a big improvement on asking nicely, but it had a gap: valid JSON isn't the same as the JSON you wanted. The model could still leave out a field you needed or add one you didn't, and you still had to tell it in the prompt to produce JSON at all.

On 6 August 2024, OpenAI announced Structured Outputs, which closed that gap. Developers could now supply a JSON Schema and set it to strict, and the model's output would follow that schema. OpenAI reported that on its own test of following complicated schemas, its new model with Structured Outputs scored 100 percent, while the older GPT-4 model from June 2023 scored less than 40 percent when relying on instructions alone. It's worth reading that figure correctly: it's OpenAI's own benchmark, it measures shape rather than accuracy, and the older model was never designed for this. But it captures the jump from "usually" to "always". OpenAI's documentation now recommends Structured Outputs over JSON mode wherever possible, and notes the exceptions: the model can still refuse a request for safety reasons (and says so in a separate field), and an answer cut off by a length limit can be incomplete.

The open-source route: grammars and Outlines (2023)

The strict approach didn't start with the big companies. In July 2023, two open-source projects showed how to do it with models anyone could run on their own computer.

On 24 July 2023, llama.cpp, a popular free program for running language models on ordinary laptops, merged a feature by developer Evan Jones called grammar-based sampling. You write a small set of rules (a grammar) describing what text is allowed, and at every step the program blocks any token that would break the rules. The examples that came with it included valid JSON, chess moves, and arithmetic. The same month, Brandon Willard and Rémi Louf published a paper, "Efficient Guided Generation for Large Language Models", describing a fast way to do the same thing, along with a free library called Outlines. The team behind Outlines went on to form a company, .txt (pronounced "dot text"), which still maintains it.

This matters to non-programmers for one reason: it shows that strict structured output isn't a proprietary trick of any single company. It's a general technique, available in free tools as well as paid services, and it's why you'll now find a structured output setting in many AI products.

Anthropic adds strict structured outputs to Claude (November 2025)

On 14 November 2025, Anthropic launched structured outputs on the Claude Developer Platform as a public beta, first for its Sonnet 4.5 and Opus 4.1 models, with more models added since. According to Anthropic's announcement, the feature uses constrained decoding so that responses conform to a supplied JSON schema or tool definition, and it reached general availability on 4 February 2026. It comes in two forms: JSON outputs, which fix the shape of Claude's answer, and strict tool use, which guarantees that when Claude asks to use a tool, the request is in exactly the right form.

Before this, the common way to get dependable JSON from Claude was careful prompting, often combined with prefilling: starting Claude's reply for it with an opening curly bracket, so the only natural continuation was the rest of the JSON. That worked well, but it was still steering rather than enforcement. Anthropic's documentation for the new feature is also a useful reminder that strict modes have limits: at the time of writing it lists schema features that aren't supported, such as minimum and maximum values for numbers, and notes that the first request with a new schema is a little slower while the system prepares it. Details like these change, so check the current documentation before relying on them.

"Let Me Speak Freely?": does forcing a format make AI think worse? (2024)

In August 2024, researchers from the Taiwanese company Appier and National Taiwan University (Zhi Rui Tam and colleagues) published a study titled "Let Me Speak Freely?". It was later presented at the EMNLP 2024 conference. They tested several models on reasoning tasks, such as maths word problems, with and without format restrictions like JSON, and reported that reasoning ability declined noticeably under format restrictions, with stricter formats generally doing more harm. They also found the opposite on classification tasks (sorting things into categories), where a fixed format sometimes helped. Their suggested fix was a two-step approach: let the model answer in natural language first, then convert that answer into the required format.

The paper was widely shared, and it drew a direct response. Later in August 2024, Will Kurt of .txt published "Say What You Mean", arguing that the study compared unlike with unlike: the structured and free-text versions used different prompts, the structured prompts left out useful task information, and the "structured" condition was mostly JSON mode rather than true schema-enforced generation. When .txt re-ran three of the tasks with matched prompts, structured answers scored the same or slightly better (for example 0.78 against 0.77 on the maths word problems). A separate 2025 benchmark study, JSONSchemaBench, by researchers including members of Microsoft's Guidance team, also reported that constrained decoding modestly improved accuracy on some of the same reasoning tasks.

The fair summary is that both sides have a point. How you ask for structure matters a great deal. A format that leaves the model no room to work through a problem can hurt, while a well-designed schema that includes space for reasoning, or the two-step approach, does no measurable harm in the tests so far. The practical lessons appear under "Common mistakes" below.

An everyday scenario: the author's reader-mail spreadsheet

Imagine Tom, a self-published thriller author who gets around forty reader emails a week. Some are fan mail, some report typos, some ask when the next book is out, and a few are review copies he's promised to follow up. For a year he's meant to track them properly and never has.

He tries asking a chatbot to "summarise these emails", and gets forty friendly paragraphs, which is no better than the inbox. So he changes the request to a form: for each email, give him one row with the sender's first name, which book it's about, a type chosen only from Fan mail, Typo report, Question, Review, or Other, the page or chapter if a typo is mentioned, and yes or no for "needs a reply". He asks for the result as CSV and pastes it into his spreadsheet. On the first run, two rows have a type of "Compliment", which isn't one of his choices, so he adds "use only these five types, exactly as written" and it doesn't happen again. Within a month he can sort by book and see every typo report for his second novel in one place, ready to send to his proofreader. Nothing about the AI changed. The form did.

Try it yourself

You need any chatbot (ChatGPT, Claude, Gemini, or similar; the free versions are fine for this) and a spreadsheet such as Google Sheets or Excel. This exercise uses the "asking nicely" approach, which is what chat windows offer, and adds the checking step that makes it trustworthy.

  1. Collect some messy text. Choose something you'd genuinely like in a spreadsheet: ten enquiry emails, a pile of reviews, a list of books with blurbs, or notes from a networking event. Remove anything private you'd rather not paste into an AI service, such as phone numbers or home addresses.
  2. Design your form first. On paper, write down the columns you want, what goes in each, and, for any column with fixed choices, the exact list of choices. Decide what should happen when the information simply isn't there. Leaving the cell blank is usually best, because it stops the AI filling gaps with guesses.
  3. Paste this prompt, with your own columns:
Turn the text below into a table in CSV format that I can paste into a spreadsheet.

Columns, in this order:
1. name: the person's first name
2. product: the product or book they mention
3. type: exactly one of Question, Complaint, Order, Praise, Other
4. urgent: yes or no (yes only if they mention a deadline or a problem with an order)
5. summary: one short sentence, no commas

Rules:
- The first line is the column names.
- One row per email.
- Use only the choices listed for "type", written exactly as shown.
- If a value is not in the text, leave that cell empty. Do not guess.
- Output only the CSV. No introduction, no explanation, no code formatting.

Text:
[paste your emails here]
  1. Load it into your spreadsheet. Copy the result into a plain text file saved with a .csv ending, then open or import it (in Google Sheets: File, Import, Upload). Pasting directly into the first cell also often works, depending on your spreadsheet; if everything lands in one column, look for a "split text to columns" option.
  2. Check it, every time. This is the step that matters. Count the rows against the number of emails. Filter the "type" column and look for anything not on your list. Pick three rows at random and compare them against the original emails, especially the empty cells: was the information really missing, or did the AI miss it? Asking nicely is usually right, and "usually" is why you check.
  3. Try JSON as well. Ask for the same information in the other format and compare. JSON is less pleasant to read, but it's what you'd hand to an automation tool or a developer.
Give me the same information as a JSON list, one object per email, with the fields
name, product, type, urgent (true or false) and summary.
If a value is missing, use null. Output only the JSON.
  1. Let it think first on hard tasks. If the task needs judgement, such as deciding whether a review is sarcastic, try asking the model to reason in plain language first, then give the table at the end, and copy only the table. This is the two-step approach from the research above.
  2. When you're ready for the real thing. If you find yourself doing this every week, that's the moment to look at the strict version: the structured output settings in OpenAI's or Anthropic's API, or the structured output options in no-code automation tools that connect to them. Ask a developer, or check your automation tool's help pages, since features and plans change and vary between services.

Common mistakes and misunderstandings

Thinking correct shape means correct content. This is the big one. Strict structured output guarantees that the price field exists and contains a number. It doesn't guarantee the number is right. A model forced to fill in a field it has no information for may simply make something up, neatly formatted. Structured output prevents broken formatting, not hallucination. Always allow "unknown" or an empty value, and check a sample by eye.

Confusing JSON mode with strict schemas. Several services have offered a setting that only promises valid JSON, meaning the brackets are in order but the fields might not be the ones you asked for. A schema-enforced mode is stronger. If a tool says it "supports JSON", find out which one it means.

Believing a prompt instruction is a guarantee. "Reply only in JSON" works most of the time, which is exactly what makes it dangerous in an automated process. The one run in fifty that adds "Here's your JSON!" at the top can stop an automation or silently drop a row. If a program depends on the format, either use an enforced mode or have the program check the format and try again when it fails.

Leaving no room to think. A schema with a single "answer" field asks the model to jump straight to the conclusion. For simple extraction that's fine. For judgement calls or anything involving reasoning, either add a field for brief reasoning before the answer field (order matters, because the model writes fields in sequence) or let it work in plain text first and convert afterwards. This is the most practical takeaway from the "Let Me Speak Freely?" debate.

Forgetting to give an escape route. If your "type" field allows only Question, Complaint, or Order, what should the model do with a spam email? With no "Other" option, it has to pick the least-wrong choice, and you'll never know. Add an "Other" or "Unclear" option to every fixed list.

Asking for too many fields at once. A form with thirty fields, several of them overlapping, produces worse answers than one with eight clear ones. Start small, and use prompt chaining if you need many different kinds of information: one step to classify, a second step to extract details.

Vague field names. A field called date in an email about an event could mean the date sent, the event date, or the reply deadline. Name fields precisely (event_date) and describe each one in a sentence, because the model reads the names and descriptions as instructions.

Using it where prose is the point. A blog post, a story, or a heartfelt customer reply doesn't belong in a form. Structured output is for information that will be sorted, counted, filtered, or passed to software. For writing that people read, let the model write.

Assuming every feature works everywhere. Strict modes support only part of the full JSON Schema standard, and the unsupported parts differ between services. At the time of writing, for example, Anthropic lists number ranges and text length limits among the unsupported constraints. Check the current documentation for whichever service you use.

Related elements