Aiodic

Back to the table

51 Intermediate Su Summarization Applications

Summarization

Condensing long content into key points.

What it is

Summarization is asking an AI to take something long and give you back something short that keeps what matters. A 60-page report becomes a page. An hour-long meeting transcript becomes five decisions and three action items. A hundred reader reviews become "people love the dialogue and find the middle slow". A 90,000-word manuscript becomes the one-page synopsis an agent asked for.

It is one of the most common things people use AI for, and one of the most useful, because it attacks a problem everybody has: too much to read and not enough time. It is also one of the easiest jobs to get subtly wrong, because a summary is only useful if you trust it, and you usually ask for one precisely because you haven't read the original.

There are two basic ways to make a summary, and the difference explains a lot about how AI summaries fail.

The idea of a machine doing this is older than most people assume. In 1958, Hans Peter Luhn, a researcher at IBM, published "The Automatic Creation of Literature Abstracts" in the IBM Journal of Research and Development. His program ran on an IBM 704 computer, counted how often significant words appeared, scored each sentence by how many of those words it contained, and printed the highest-scoring sentences as an "auto-abstract". That was purely extractive, and the basic idea of scoring and picking sentences dominated the field for decades. Today's language models are abstractive by nature, which is why they're so much more flexible, and why checking them matters so much more.

Beyond extractive and abstractive, summaries come in different shapes for different jobs. It helps to name the shape you want, because "summarise this" on its own leaves the model to guess.

Why it matters

It is how most people now meet AI. Summaries show up without being asked for: at the top of search results, in email apps, in meeting tools that write up the call, on phone lock screens, and in chatbots answering questions about the news. Many people read the AI's summary of something far more often than they read the thing itself. That makes the quality of summaries a matter of public trust, not just personal convenience.

It saves real hours for creators. A non-fiction author can summarise thirty research papers to decide which five to read closely. A podcaster can turn a transcript into show notes. A novelist can get a chapter-by-chapter outline of their own draft to spot where the middle sags. A small publisher can condense a pile of reader reviews into themes. None of these replace reading, but all of them help you decide what to read and where to look.

You control the angle, and the angle is most of the value. A good human assistant doesn't give you "a summary"; they give you the summary you need. The same 40-page market report can be summarised for your accountant (costs and risks), your cover designer (what the genre's bestsellers look like), or yourself (should I write this book at all?). Modern models are very good at taking an angle when you name one, and much less useful when you don't.

A summary decides what you never see. This is the part people forget. Every summary is a set of choices about what to leave out, and when a model makes those choices, you inherit its judgement without seeing it. If the summary skips the one objection that should have changed your mind, you won't know, because you didn't read the original. That's why summarization belongs on the same shelf as hallucination and grounding: the risk isn't only that the AI makes something up, it's also that it quietly drops something true.

Summaries have their own typical failures. Knowing them lets you ask better and check faster:

Real-world examples

Apple's lock-screen summaries and the headline that never ran (December 2024 to January 2025)

In late 2024, Apple added AI notification summaries to iPhones as part of Apple Intelligence. When several alerts arrived from one app, the phone condensed them into a single short line. In December 2024, a summary of BBC News alerts told users that Luigi Mangione, the man charged over the killing of UnitedHealthcare's chief executive, had shot himself. He hadn't. The BBC complained to Apple, pointing out that the false line appeared under its name. Other summaries went wrong too; one bundle of New York Times alerts was condensed into a line saying the Israeli prime minister had been arrested, when the story was about an International Criminal Court arrest warrant.

On 16 January 2025, Apple switched off notification summaries for news and entertainment apps in test versions of its next software update, while it worked on improvements. It also said summaries would appear in italics so people could tell them apart from real headlines, and added a warning that the feature was in beta and could contain errors.

The lesson for anyone summarising is about length and context. A single line has almost no room for qualifiers, and a model squeezing several stories into one sentence has every opportunity to blend them. The shorter you ask a summary to be, the more carefully you need to check it.

The broadcasters who graded AI answers about the news (February and October 2025)

In February 2025, the BBC published research in which its journalists checked how four AI assistants (ChatGPT, Microsoft Copilot, Google Gemini and Perplexity) answered 100 questions about the news, with the assistants told to use BBC News articles as sources. The journalists judged that 51 percent of the answers had significant issues of some kind. Of the answers that cited BBC content, 19 percent introduced factual errors such as wrong numbers or dates, and 13 percent of quotes attributed to BBC articles had been changed or didn't appear in the article cited. Among the examples reported: assistants describing Rishi Sunak and Nicola Sturgeon as still in office after they had left, and an answer saying the NHS advises people not to start vaping in order to quit smoking, when the NHS actually recommends vaping as a way to quit.

In October 2025, the European Broadcasting Union and the BBC published a much larger follow-up, "News Integrity in AI Assistants". Journalists from 22 public service broadcasters in 18 countries, working in 14 languages, reviewed more than 3,000 answers from the same four assistants. They found that 45 percent had at least one significant issue, and sourcing problems, such as claims attributed to the wrong outlet or not supported by the source cited, were the single biggest cause.

These studies tested answers built from news articles rather than a plain "summarise this document" request, but that's what makes them relevant: an answer about the news is a summary of other people's reporting. Both reports describe summaries that sound authoritative, cite a real source, and still don't say what the source says.

The regulator that tested AI against its own staff (2024)

In early 2024, Australia's corporate regulator, the Australian Securities and Investments Commission (ASIC), ran a five-week proof of concept with Amazon Web Services. The job was a realistic one: summarise public submissions to a parliamentary inquiry, focusing on mentions of ASIC, recommendations, and references to more regulation, with page references. After trying several models, the team used Meta's Llama2-70B. ASIC staff did the same task by hand, and reviewers graded all the summaries without being told which were written by the AI.

The human summaries scored 81 percent on the grading rubric and the AI summaries 47 percent. According to the report, which reached the public through a Senate committee and was reported by Crikey in September 2024, the AI summaries often missed emphasis, nuance and context, sometimes included incorrect information or left out relevant information, and sometimes focused on side points. The reviewers concluded that such summaries could create more work, because someone would need to check them against the original submissions.

Llama2 is an old model now, and newer ones do considerably better, so don't read the scores as today's numbers. The pattern is what lasts: summaries that looked fine at a glance but lost the nuance, and the discovery that checking a bad summary can take longer than writing a good one.

Summarising whole books, and checking them claim by claim (2021 and 2024)

In September 2021, OpenAI published "Recursively Summarizing Books with Human Feedback". The model summarised small sections of a novel, then summarised those summaries, and so on up to a summary of the whole book. It built on the same team's earlier work training models to summarise Reddit posts using human preference ratings, described in the RLHF article. The results were promising but modest: by the paper's own account, the summaries matched the quality of human-written ones only in a few cases, about 5 percent of books.

In 2024, researchers from the University of Massachusetts Amherst and others published FABLES, a study built to answer the question every author should ask: are AI summaries of a novel actually true? They used books published in 2023 and 2024, so the models couldn't have memorised existing summaries, and they hired readers who had finished each book to check 3,158 claims made in AI summaries of 26 novels. Most of the errors involved events and the state of characters, and spotting them usually took knowing the story well, not just skimming. They also found that AI checkers were poor at catching these mistakes. The lesson for a writer asking for a synopsis of their own manuscript is encouraging and sobering at once: you're the ideal fact-checker, because you've read it, and you should check it.

An everyday scenario: the research pile for a non-fiction book

Imagine Tomás, who is writing a popular history of coffee and has forty saved articles and papers he hasn't had time to read. He pastes them into a chatbot one at a time and asks for a three-sentence summary of each. The summaries are smooth and he feels productive. Then, drafting chapter three, he quotes a figure "from" one paper and, on a whim, opens the original. The number is there, but it describes a single port in a single year, not the whole trade. The summary had dropped the qualifier.

He changes his approach. For each source he now asks for two things: a short summary for deciding whether to read it, and a separate list of every specific claim, with the sentence it comes from quoted exactly. Anything he plans to use in the book, he checks against the quoted sentence. It takes longer per source, but he stops building chapters on details that aren't quite what the source said.

Try it yourself

You need a long article you can read yourself (3,000 words or more is ideal; a long magazine feature or a report works well) and any AI chatbot. Allow half an hour. The aim is to see how much the angle changes a summary, and to practise checking one.

  1. Pick the article and read it once yourself. Yes, the whole thing. You can't judge a summary of something you haven't read, and this exercise is about learning what summaries leave out.
  2. Paste the full text into a new chat. Pasting is better than giving a link, because some tools can't open links reliably or only read part of the page. If the article is very long, check that nothing got cut off at the end.
  3. Ask for a summary for a busy friend. Copy and paste this, filling in the brackets:
Summarise the article below for a busy friend who has two minutes.
Give the single most important point in one sentence, then three
short bullet points with the key supporting details.
Use only what the article says. If the article hedges or disagrees
with itself, keep that uncertainty in the summary.

Article:
[paste the article]
  1. In the same chat, ask for a fact-checker's list. This is a completely different summary of the same text:
Now list every specific factual claim in the article: names, numbers,
dates, quotes, and cause-and-effect statements. For each one, quote
the exact sentence from the article it comes from. Don't include
anything that isn't directly stated in the article. Number the list.
  1. Compare the two. Notice what the friendly summary chose to keep and what it left out. Look at the claims list and ask: were any important claims missing from the friendly version? Is anything in the friendly version not on the claims list at all? That second case is a warning sign of an invented or overstated detail.
  2. Spot-check three claims. Pick three items from the claims list, ideally one from the beginning, one from the middle and one from the end of the article. Use your browser's find function (Ctrl+F on Windows, Cmd+F on a Mac) to search the original for a distinctive word from each quoted sentence. Check that the quote really exists and that the claim means what the summary says it means. Pay special attention to the middle one.
  3. Try a different angle. Ask for the same article summarised for a specific reader you care about, such as "a reader of my newsletter about small business" or "someone deciding whether to cite this in a book". Watch how the choices change.
  4. Ask what's missing. This prompt often surfaces what the summaries dropped:
Compare your first summary with the full article. List the three most
important points, caveats, or counterarguments that the summary left
out, and explain in one line each why a reader might need them.

If the answer to step 8 surprises you, that's the lesson. Every summary is a selection, and now you've seen what this one's selection cost.

A few habits to carry forward once you've done this once:

Common mistakes and misunderstandings

Treating a summary as a substitute for the source when it matters. A summary is a map, not the territory. It's perfect for deciding what to read, triaging a pile of documents or refreshing your memory of something you've already read. It's risky as the only basis for something you'll publish, sign or act on. The more a decision depends on a detail, the more that detail should be checked in the original.

Asking for "a summary" without saying what for. Without an angle, the model guesses at what's important, and its guess is often whatever the source repeats most. Tell it who the reader is and what they need to decide. You'll get a shorter, more useful summary.

Assuming shorter is always better. The tighter the length limit, the more the model has to cut, merge and generalise, which is exactly where nuance flattens and details blur. A one-line summary of a complex report is almost guaranteed to lose something that matters. Pick the length that fits the job, not the smallest one possible.

Believing it can't invent things because the source is right there. Having the document in front of it reduces invention a lot, but it doesn't eliminate it. Leaderboards that test exactly this still find every model adding unsupported details some of the time. Grounding helps, and asking for quotes helps more, but only checking catches what slips through.

Not noticing when the tool only read part of the document. Some chat tools and apps quietly truncate long pastes or uploads, summarise only the first part of a web page, or can't read scanned PDFs properly. If a summary of a long document seems to cover only the opening, ask the model what the last section says. If it can't tell you, it didn't see it.

Summarising your own manuscript and trusting the result. Authors increasingly ask AI for synopses, blurbs and chapter outlines of their own books. It's a good use, and you're uniquely placed to check it, but check you must. The FABLES study found the typical error in book summaries is getting events or a character's situation wrong, which is exactly what would embarrass you in a query letter.

Letting the summary replace the voice. Summaries are flat by design. If you use one as the basis for a blurb, a pitch or show notes, rewrite it in your own voice. The writing assistance article covers how to do that without losing what makes your writing yours.

Pasting private material into a tool without checking the terms. Meeting transcripts, client documents, unpublished manuscripts and reader emails can be sensitive. At the time of writing, AI services differ in whether they keep your inputs or use them for training, and plans and settings change. Check your tool's current privacy terms before summarising anything you wouldn't want stored.

Trusting an AI to check an AI's summary. It's tempting to ask a second chatbot "is this summary accurate?", and it can catch obvious problems. But research on book-length summaries found automated checkers were unreliable at spotting the subtle errors, which are the ones that matter. Use AI to produce a claims list you can check quickly, and do the final check yourself.

Related elements