A chatbot can write you a poem in a few seconds. Getting it to that point took years of work, mountains of data, and a lot of people checking its answers.
That whole process lives in one group on the Periodic Table of AI: Data & Training, the yellow-green tiles. There are six of them, and together they answer a simple question. How does a model get from knowing nothing to being useful?

If you want the background first, the last post on Machine Learning covers the main ways computers learn. This one is about what they learn from and how they get checked.
What the model learns from
Datasets
A dataset is a collection of examples. For a model that sorts photos, it’s a pile of pictures with labels. For a chatbot, it’s a huge pile of text.
One famous example is ImageNet. Computer scientist Fei-Fei Li began working on the idea in 2006, and her team showed it publicly in 2009. It grew to more than 14 million images, each labeled with what it shows. A smaller slice, about 1.2 million training images across 1,000 categories, became the basis for a yearly contest where teams competed to build the best image-recognition system. Li’s bet was that the scale and variety of the data mattered as much as clever design.
Text models have their own giant source. Common Crawl is a nonprofit that has been collecting web pages since 2008 and gives the data away for free. It holds hundreds of billions of pages now, and filtered versions of it have been a main ingredient in many large language models.
The big lesson is that a model can only learn what’s in its data. If the data has gaps or leans one way, the model will too.
Pretraining
Pretraining is the first big stage. The model reads an enormous amount of text and practices one job: guess the next token. Each wrong guess nudges its parameters a little, and this repeats an almost unimaginable number of times. It takes huge amounts of computing power.
The name is hiding in plain sight. GPT stands for Generative Pre-trained Transformer, and the “pre-trained” part is this stage.
Pretraining gives you a model that knows a lot about language. It doesn’t give you a good assistant. A model trained only to predict the next word isn’t guaranteed to be helpful, truthful, or good at following instructions. The next two tiles fix that.
How the model gets shaped
Fine-tuning
Fine-tuning means taking a pretrained model and training it some more on a smaller, focused set of examples. Think of a new hire with a good general education who then gets job training.
OpenAI’s InstructGPT is a clear example. It started as GPT-3 and was fine-tuned on examples written by people. It’s the same idea as Transfer Learning from the last post: start with a model that already knows a lot, then teach it your job.
RLHF
RLHF stands for reinforcement learning from human feedback. OpenAI used it for InstructGPT, and it works in three steps.
First, people write examples of good answers, and the model is fine-tuned on them. Second, the model writes several answers to the same prompt, and people rank them from best to worst. Those rankings train a separate “reward model” that learns to predict which answers people would prefer. Third, the main model practices with reinforcement learning, using the reward model’s score as its reward.
The results were striking. In OpenAI’s tests, people preferred answers from a 1.3 billion parameter InstructGPT over answers from the 175 billion parameter GPT-3, even though the small model had about 100 times fewer parameters. OpenAI made InstructGPT the default on its API in January 2022. The technique wasn’t brand new, either. OpenAI’s paper cites earlier work from 2017 and 2020.
There’s a catch. The feedback comes from people, so the model picks up their preferences. OpenAI itself pointed out that InstructGPT was trained to follow instructions in English, so it leans toward the cultural values of English speakers. That’s one way Bias gets in.
Fresh data and report cards
Synthetic Data
Synthetic data is data made by a computer instead of collected from the real world. Teams use it to fill gaps, like rare situations where real examples are hard to find. Often, other AI models generate it.
There’s a risk. In a 2024 paper in the journal Nature, Ilia Shumailov and colleagues tested what happens when models are trained on content produced by earlier models. They found that careless use of that content causes lasting damage. The rare, unusual material in the original data disappears. They call it model collapse. It works like a photocopy of a photocopy, where each round loses a little more detail.
The authors say training on AI-made data isn’t impossible, but the data has to be filtered carefully. They also point out that data from real human interactions gets more valuable as more of the web fills up with AI-generated text.
Benchmarks
Benchmarks are standardized tests for AI models. Every model answers the same questions, and you compare the scores.
A well-known one is MMLU. It’s a multiple-choice test covering 57 subjects, from elementary math to law, created by Dan Hendrycks and colleagues. The paper came out in 2020 and was presented at a conference in 2021.
Tests have limits. Top models now score close to the ceiling on MMLU, so it can’t separate them very well, and a harder version called MMLU-Pro was built for that reason. The questions are also public, which creates a risk that they leak into training data. And a high score only tells you how a model does on that test. It doesn’t tell you how it will do on your job.
How they fit together
Datasets supply the raw material. Pretraining turns it into a model that knows language. Fine-tuning and RLHF shape that model into something that helps. Synthetic data can add more material when used with care, and benchmarks check how it all turned out.
Where to go next
Once a model is trained, the next question is how to talk to it. The next group on the chart is Prompting, and it’s about getting better answers out of the models you already use.
In the meantime, click any tile above and read the full page. If you only have a couple of minutes, start with Datasets, since everything else in this group depends on it.
1 thought on “AI Training Data: How Models Get Fed, Tuned and Tested”
Comments are closed.