Aiodic

Back to the table

30 Intermediate Mp Meta-prompting Prompting

Meta-prompting

Using AI to write or improve prompts.

What it is

Meta-prompting means using an AI to help write the instructions you give an AI. Instead of staring at a blank box wondering how to phrase your request, you tell the model what you're trying to achieve and ask it to draft the prompt for you. Or you paste in a prompt you already use and ask what's wrong with it. Or you hand it a few examples of good and bad results and ask it to rewrite the prompt so the bad ones stop happening.

"Meta" is a word borrowed from Greek that, in English, usually means "about itself". A meta-prompt is a prompt about prompts. The idea sounds circular at first, and it's fair to ask why a model would be any better at writing instructions than you are. There are two good reasons.

The first is that the model has read an enormous amount of writing about how to instruct people and machines: manuals, briefs, style guides, and a great deal of public discussion about prompting itself. It tends to remember the things beginners forget. Who is the audience? What format should the answer take? What should happen when the input is missing something? What does "good" look like? A well-asked model will often add those details without being told.

The second is that the model is a good interviewer. The biggest weakness of most prompts isn't the wording. It's that the person writing them hasn't decided what they actually want. Ask a model to question you before it drafts anything, and it will pull out the decisions you've been skipping: how long, how formal, what to leave out, and what to do with the awkward cases.

In everyday use, meta-prompting comes in three flavours:

Researchers took the same idea much further. Instead of one person asking one chatbot for a better prompt, they built systems in which a model proposes dozens or hundreds of candidate prompts, each one is scored automatically on a set of test questions, and the best ones are fed back to the model to inspire the next round. That's called automatic prompt optimisation, and it's where some of the most surprising prompts ever written came from. Two famous examples follow below.

A note on the name, because you may meet it used differently. In January 2024, Mirac Suzgun of Stanford and Adam Tauman Kalai of OpenAI published a paper called "Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding". In their method, a single model plays two parts. As a "conductor", it breaks a hard problem into pieces and writes instructions for each piece. It then hands each piece to fresh copies of itself, set up as "experts" for that piece, and finally checks and combines their answers. Working with GPT-4, and with access to a Python interpreter (a tool that runs code), they reported that this beat ordinary prompting by 17.1 percent on average across the tasks they tested, which included maths puzzles and chess problems. That's a real and influential technique, and it sits closer to orchestration and multi-agent systems than to the everyday habit this page is mostly about. When people say "meta-prompting" in a newsletter or a course, they usually mean the everyday version: getting the AI to help with your prompts.

Why it matters

It's the fastest way to get better at prompting. When a model rewrites your prompt, you can see exactly what it added: a line about the audience, a rule for missing information, a sample of the output format. Each addition is a small lesson. People who use meta-prompting regularly tend to find, after a few weeks, that their own first drafts start to include those things without being asked. That slow upgrade to your own skill is the most lasting benefit.

It fills the gaps you can't see. You know your business, so you leave out things that feel obvious to you. The model doesn't know them, so it guesses. A critique request is a cheap way to find those gaps, because the model is good at saying "this prompt doesn't say what to do if the customer asks about something you don't sell."

It saves time on long or technical prompts. A system prompt for a customer-facing assistant, or a prompt template that will run hundreds of times, can easily run to a page. Drafting that page from scratch is slow. Having a model produce a first version from a conversation, and then editing it, is usually much faster.

It turns fixing into a routine. Most prompt problems show up as a bad output. Meta-prompting gives you a standard move: paste the prompt, paste the bad output, say what was wrong, and ask for a revised prompt. That's the core of iterative refinement, and the model does the rewriting while you do the judging.

But it only matters if you test the result. This is the part that gets skipped. An AI-written prompt looks impressive. It's long, well organised, and full of sensible rules, and that polish makes it easy to trust. It doesn't mean it works better. A rewritten prompt can fix the problem you pointed at and quietly break something you didn't mention. The research systems described below never trusted a new prompt on looks: every candidate was scored on test questions before it was kept. Anthropic's announcement of its prompt improver pointed people straight at a tool for testing prompts against ideal answers, and OpenAI's documentation for its prompt optimizer tells developers to evaluate and review optimised prompts before using them, warning that the new prompt may do worse than the original on some inputs. For everyone else, the same discipline has a name: evals. A handful of real test inputs, run on the old prompt and the new one, tells you whether the AI's rewrite is an improvement or only a makeover.

Real-world examples

The model that wrote a better "think step by step" (November 2022)

By 2022, researchers had found that adding "Let's think step by step" to a maths question made language models noticeably better at it, the idea behind chain of thought. That phrase had been written by people. A team from the University of Toronto and the Vector Institute, led by Yongchao Zhou and including Jimmy Ba, asked whether a model could find a better one.

Their paper, "Large Language Models Are Human-Level Prompt Engineers", first posted on 3 November 2022, described a method they called Automatic Prompt Engineer, or APE. A model was shown examples of a task and asked to propose instructions that would produce those results. Each proposed instruction was then tried on the task and scored, and the best scorers were kept. Across 24 language tasks, the instructions the model wrote matched or beat the ones written by people on 19.

The most quoted result was a replacement for "Let's think step by step". APE's search, using OpenAI's InstructGPT model, landed on "Let's work this out in a step by step way to be sure we have the right answer." On a set of arithmetic word problems called MultiArith, that nudged the score from 78.7 to 82.0, and on the harder GSM8K school maths set from 40.7 to 43.0. Small gains, but found by a machine, and better than the phrase the whole field had been copying. The point for everyday users: the winning prompt wasn't picked because it read well. It was picked because it scored best on a test.

"Take a deep breath" (September 2023)

The next famous result came from Google DeepMind. In a paper titled "Large Language Models as Optimizers", first posted on 7 September 2023, Chengrun Yang and colleagues described a method called OPRO, short for Optimization by PROmpting. The model is given a list of the prompts tried so far, each with its score, and asked to write a new one that should score higher. The new prompt is tested, its score is added to the list, and the loop repeats.

When they pointed this at GSM8K, the maths word problems again, with Google's PaLM 2 model, the best instruction it found was "Take a deep breath and work on this problem step-by-step." On their test that scored 80.2 percent, against 71.8 percent for the human-written "Let's think step by step". Other optimiser models produced their own winners, such as "Break this down", which scored 79.9 percent in one setup. Overall, the paper reported that OPRO's prompts beat human-designed ones by up to 8 percent on GSM8K and by up to 50 percent on some tasks from a harder collection called Big-Bench Hard.

The "deep breath" line went viral, and a lot of people started adding it to every prompt as a magic phrase. That misses the lesson. The phrase was the best of many candidates for one model on one kind of question, chosen by scoring. Nothing in the paper suggests it helps a different model write your marketing copy. What transfers is the method: generate options, test them, keep what scores.

AI companies build prompt writers into their own tools (2024 onwards)

Once the research showed models could write good prompts, the AI companies turned the idea into buttons. On 20 May 2024, Anthropic announced a prompt generator in its developer console, the website developers use to build with Claude: describe the task, and Claude drafts a detailed prompt template. Anthropic's own announcement was candid that the generated prompts "do not always produce perfect results", while saying they often beat hand-written prompts from people new to prompt engineering. On 14 October 2024 it added a prompt improver that rewrites an existing prompt, restructuring it, adding worked examples, and asking the model to reason before answering. Anthropic reported that in its tests the improver raised accuracy by 30 percent on a classification task and brought word-count adherence to 100 percent on a summarisation task.

OpenAI followed with a prompt optimizer in its developer dashboard, described in its documentation as a tool that rewrites a prompt according to current best practices, fixing things like contradictions and unclear formatting. It can also use a set of test examples and grades to guide the rewrite, and the documentation says plainly that the results depend on the quality of those grades, and that optimised prompts should always be evaluated and reviewed before use. At the time of writing, OpenAI has announced that the version tied to its evals platform is being retired, which is a good reminder that these tools change often. Check what's currently offered, and on what terms, before building on one.

For developers there is also DSPy, a free, open-source framework from Stanford's natural language processing group, first described in a paper in October 2023. Its own description calls it a framework for programming language models rather than prompting them. You describe what goes in and what should come out, supply examples and a way to score results, and DSPy's optimisers search for the instructions and examples that score best. It's a technical tool, but it's the same idea as everything above: let the machine write the prompt, and let a score decide.

An everyday scenario: the podcast show notes

This is an illustrative scenario, not a documented event. Marcus hosts a weekly interview podcast and wants AI to turn each transcript into show notes. His prompt, "Write show notes for this episode", gives him something different every week: sometimes a list, sometimes three paragraphs, once a summary that misspelled the guest's company.

He tries meta-prompting. First he asks the chatbot to interview him about what good show notes look like. It asks who reads them, how long they should be, whether he wants timestamps, how to handle guest names and links, and what tone he uses. From his answers it drafts a prompt with a fixed structure: a two-sentence hook, five timestamped highlights, and a rule to copy names exactly as they appear in the transcript and flag anything uncertain.

Then he tests it on his last three episodes, including one with a guest who spoke in long tangents. Two come out well. The third invents a timestamp. He pastes the prompt and that output back and asks what caused it; the model points out that the transcript had no timestamps for that section, and suggests a rule to write "no timestamp available" instead of guessing. He adds the rule, runs all three again, and all three pass. The model wrote most of the prompt. The three test episodes are why he trusts it.

Try it yourself

You need a chatbot and one task you'd like the AI to do regularly. Allow thirty minutes. Use the same chatbot throughout, and start a fresh chat whenever the steps say so.

  1. Pick one task. Something repetitive and specific: turning notes into a weekly update, writing product descriptions, replying to one type of customer email. Gather three real inputs for it now; you'll need them in step 5.
  2. Let the AI interview you. In a new chat, paste: “ I want to write a prompt that I can reuse for this task: [describe the task in one or two sentences]. Before you write anything, interview me. Ask me one question at a time about the audience, the purpose, the format, the length, the tone, what to avoid, and what should happen when the input is unclear or missing something. Stop after about eight questions, or sooner if you have enough. “ Answer honestly, including "I don't know yet". Those are the decisions you've been skipping.
  3. Ask for the draft. When the questions stop, paste: “ Now write the prompt. Make it self-contained so it works in a fresh chat with no memory of this conversation. Put the instructions first and leave a clearly marked place at the end where I paste the input. Keep it under 300 words. “
  4. Ask for a critique before you trust it. Start a new chat, so the model isn't defending its own work, and paste: “ Here is a prompt I plan to reuse. Act as a critical reviewer. List up to five specific problems: anything unclear, contradictory, missing, or likely to produce inconsistent results. For each problem, suggest a one-line fix. Do not rewrite the whole prompt. [paste the prompt] “ Accept the fixes you agree with and ignore the rest. You're the editor.
  5. Test it on your three real inputs. Open a fresh chat for each input, paste the prompt, add the input, and save the result. Before reading them, write down two or three checks that any good answer must pass, such as "under 150 words", "mentions the deadline", "doesn't invent a price". Then grade each result pass or fail.
  6. Feed the failures back. If something failed, paste: “ Here is my prompt, an input, and the output it produced. The output failed this check: [describe what was wrong]. Suggest the smallest change to the prompt that would fix this without changing anything else. Prompt: [paste] Input: [paste] Output: [paste] “
  7. Re-run all three, not just the one that failed. A fix for one input can break another. If all three pass, save the prompt somewhere you'll find it, with the date and the three test inputs next to it. Next time you change the prompt, run those three again first.
  8. Optional: compare with your old prompt. Run your original, one-line prompt on the same three inputs and grade those too. Seeing the two scores side by side is the quickest way to learn what the AI-written version actually added.

Common mistakes and misunderstandings

Trusting the prompt because it looks professional. AI-written prompts are long, tidy, and full of confident rules, and none of that is evidence they work. The researchers who made meta-prompting famous scored every candidate on test questions. Do the small version of the same thing: three or more real inputs, graded against written checks.

Letting the model fill in decisions that are yours. If you ask for a prompt without saying who the audience is, the model will pick an audience. It might pick well, but it's guessing, and the guess gets written into the prompt as if it were a rule. The interview step exists to make sure the important choices come from you.

Treating viral phrases as magic. "Take a deep breath" was found by testing many options on one model for one kind of maths question. It isn't a universal upgrade, and neither is any other phrase someone found. If you want to know whether a phrase helps your task, test it on your inputs.

Asking the same chat to critique its own draft. A model reviewing text it has just written in the same conversation tends to be gentle with it. A fresh chat, with an instruction to act as a critical reviewer, gives a more honest critique. This is also the idea behind self-critique, which works best when the critique is asked for separately and specifically.

Accepting ever-longer prompts. Each round of "improve this" tends to add rules and rarely removes any. After five rounds you can have a page of instructions, some of which conflict. Ask the model to shorten as well as improve, for example "cut anything that isn't changing the output", and check that the test inputs still pass.

Assuming a prompt written for one model works on another. A prompt tuned on one chatbot may behave differently on another, or on a newer version of the same one. The optimisation research was always done against a particular model. When you switch tools, re-run your test inputs.

Confusing the two meanings of the word. If you read a paper or a tutorial about "meta-prompting" that describes a conductor model handing sub-tasks to expert copies of itself, that's the Suzgun and Kalai method: a way of structuring how a model works through a hard problem. It's worth knowing about, but you don't need it to get value from the everyday habit of asking the AI to help with your prompts.

Never writing prompts yourself again. Meta-prompting is a shortcut and a teacher, not a replacement for knowing what you want. The best results come from people who understand why the AI's draft includes what it does, and who can spot when it has quietly got something wrong.

Related elements