Aiodic

Back to the table

19 Intermediate Ul Unsupervised Learning Machine Learning

Unsupervised Learning

Finding structure in unlabeled data.

What it is

Unsupervised learning is the branch of machine learning where the computer is handed a pile of data with no answers attached and asked to find the structure in it by itself. Nobody has labelled the photos "cat" or "dog". Nobody has marked the customers "loyal" or "about to leave". There is just the data, and the job is to notice what is in it: which items belong together, which features really matter, and which items don't fit at all.

Compare it with its sibling, supervised learning. There, every training example comes with the right answer, like a stack of exam papers with the answer key stapled on, and the model learns to reproduce those answers. In unsupervised learning there is no answer key at all. A good everyday picture is a jar of mixed buttons tipped onto a table. Nobody tells you the categories, but within a minute you've made piles: big ones, small ones, the shiny metal ones, the odd ones with four holes. You found the groups because they were there to be found. Unsupervised learning does the same thing with numbers, at a scale and in a number of dimensions no person could manage.

Most unsupervised learning does one of three jobs.

Clustering: finding the natural groups. The algorithm puts similar items together and different items apart. The best-known method, called k-means, is almost simple enough to do with a pencil. You choose how many groups you want (that number is the "k"). The computer drops that many markers at random among the data, assigns every item to its nearest marker, then moves each marker to the middle of the items it collected. It repeats those two steps, assign and move, until nothing changes. The idea goes back to the 1950s: Stuart Lloyd worked out the standard version at Bell Labs in 1957, though it wasn't published as a journal article until 1982, and James MacQueen gave it the name "k-means" in 1967. It is still one of the first tools a data analyst reaches for.

Dimensionality reduction: squeezing data down to what matters. Real data often has hundreds of columns: every product a customer has bought, every gene measured in a cell, every pixel in a photo. Many of those columns move together and say nearly the same thing. Dimensionality reduction (reducing the number of columns, or "dimensions") finds a much smaller set of summary measures that keeps most of the information, the way a good summary keeps the plot of a novel without every sentence. This is a kind of compression, and it's useful for two reasons. It makes data small enough to plot on a screen, so people can see its shape. And a model that has learned to compress something well has, in a real sense, learned what that thing is made of. The embeddings that let AI measure how close two meanings are come from exactly this family of ideas.

Anomaly detection: spotting what doesn't fit. Once a system has learned what normal looks like, it can flag whatever is far from normal: a card payment that looks nothing like the owner's usual spending, a sensor reading from a factory machine that has started to drift, a login from an unusual place at an unusual hour. The great strength here is that nobody needs a list of every past fraud or failure. The system doesn't know what the new problem is, only that this item is unlike everything else.

Where self-supervised learning fits

There is a fourth idea you'll hear constantly, because it is how today's chatbots are built: self-supervised learning. It sits on the border between unsupervised and supervised, and it's worth getting the distinction right.

In self-supervised learning, nobody labels the data, but the training does have right answers. The trick is that the answers are cut out of the data itself. Hide the next word of a sentence and ask the model to guess it; the real next word is the answer. Blank out a patch of a photo and ask the model to fill it in; the real patch is the answer. That is the whole of pretraining for a language model: guess the next token, trillions of times, with the text serving as its own answer key.

So is that unsupervised or not? Both descriptions are fair, depending on which part you look at. It is unsupervised in the sense that matters most in practice: no person had to label anything, so it can learn from the whole public web. It is supervised in the mechanical sense: each guess is checked against a specific correct answer. Researchers used to file it under unsupervised learning. When OpenAI introduced its first GPT model in June 2018, the blog post was titled "Improving language understanding with unsupervised learning". Today most people use the more precise term "self-supervised" for this kind of training, and keep "unsupervised" for methods like clustering, where there is no right answer to check against at all.

The short version: unsupervised learning finds structure with no answers; self-supervised learning manufactures its own answers from the raw data; supervised learning needs a person to supply the answers.

Why it matters

Most of the world's data has no labels. Your sales records, your email archive, a hospital's scan library, every word on the web: none of it arrived with neat tags saying what it means. Labelling is slow and expensive, because it needs people, and often it needs experts. Unsupervised and self-supervised methods are how AI learns from the huge unlabelled majority instead of the small labelled minority. The datasets behind modern AI are mostly raw material of exactly this kind.

It finds things you didn't know to look for. Supervised learning can only learn the categories you already named. Unsupervised learning can hand you categories you didn't know existed. That makes it a discovery tool. In medicine, grouping tumours by their genetic activity revealed types of cancer that looked the same under a microscope but behaved differently, as the breast cancer example below shows. In business, clustering customers often turns up a group the owner had never noticed, such as people who only ever buy in December, or buyers who always pair two particular products.

It is the foundation of modern AI. The chatbots, image generators, and translation tools people use every day learned most of what they know without anyone labelling it. A language model's knowledge of grammar, facts, and style comes from self-supervised pretraining. The later stages, fine-tuning and RLHF, are short and use a comparatively small amount of human-labelled data. Even the diffusion models behind image generators trace back to a 2015 paper with "unsupervised learning" in its title.

It catches the new problem, not just the old one. A fraud system trained only on past fraud (supervised) learns yesterday's tricks. An anomaly detector learns what normal looks like, so it can flag a pattern nobody has seen before. In practice, many banks and security teams combine both kinds: the anomaly detector raises its hand, and people or a supervised model decide what it means.

It makes big data small enough for people to understand. A spreadsheet with 10,000 rows and 60 columns is unreadable. A chart showing that those rows fall into five groups, each with a name and a short description, is something a team can argue about over lunch. Much of the practical value of unsupervised learning is simply turning a wall of numbers into a picture.

The catch is that the machine finds structure, not meaning. It will tell you that 1,200 customers form a group. It won't tell you why, or whether the group matters, or whether it's an artefact of how the data was collected. That part is still your job, and it is the reason unsupervised results need a sceptical human reading them.

Real-world examples

Google's network that found cats on its own (2012)

In June 2012, a team from Google and Stanford University reported an experiment that became famous for an unlikely reason: cats. The researchers, including Quoc Le, Jeff Dean, and Andrew Ng, built a very large neural network with about a billion connections and spread it across 1,000 computers with 16,000 processor cores. They fed it 10 million small still images taken from YouTube videos, each 200 by 200 pixels, and trained it for three days.

None of the images carried a label. The network's only task was to learn to compress each picture and rebuild it, which forced it to discover the features that pictures are made of. When the team looked inside afterwards, they found individual units in the network that responded strongly to human faces, others to human bodies, and one that responded strongly to cat faces. Writing on Google's official blog on 26 June 2012, Jeff Dean and Andrew Ng stressed that the network had never been told what a cat was and had never seen a single image labelled as one.

The finding mattered beyond the headlines. It showed that a large enough network, given enough unlabelled data, could teach itself the building blocks of vision. When the team then used what it had learned as a starting point for a labelled task, recognising 20,000 kinds of object in the ImageNet collection, it reached 15.8 percent accuracy. That sounds low, but it was a 70 percent relative improvement on the previous best result for such a large, hard task. The work was presented at ICML, a major machine-learning conference, that same summer. The lesson that unlabelled data can teach a machine what the world looks like is the same one that later drove pretraining.

Clustering reveals hidden types of breast cancer (2000 to 2013)

In 2000, a team led by Charles Perou and David Botstein published "Molecular portraits of human breast tumours" in Nature. They measured the activity of about 8,000 genes in breast tumour samples using a technology called a DNA microarray, a glass slide that reads thousands of genes at once. That produced more than 680,000 data points, far too many for anyone to read.

So they used hierarchical clustering, an unsupervised method that repeatedly joins the most similar samples into groups and draws the result as a branching tree. Nobody told the algorithm which tumours were which. The tree still split the cancers into distinct families: a luminal group that resembled the cells lining the milk ducts and was mostly driven by the hormone oestrogen, a basal-like group that resembled a different layer of cells and lacked the oestrogen receptor, a group with high activity of the gene ERBB2 (better known as HER2), and a group that looked like normal breast tissue. Tumours that looked alike under a microscope turned out to be different diseases at the level of their genes.

The authors were careful to say that they didn't yet have outcome data. That came in 2001, when Therese Sørlie and colleagues, working with the same group, published a study of 78 breast cancers in PNAS. It confirmed the families, split the luminal group in two, and showed that the groups had significantly different survival. The basal-like group in particular had a worse outlook. The families are now known as the intrinsic subtypes, and they entered routine care. A test called Prosigna, based on a 50-gene version of the classification known as PAM50, received US Food and Drug Administration clearance in 2013 and reports a tumour's subtype and risk of recurrence. It is one of the clearest cases of unsupervised learning finding something real that people had missed, and then humans doing the slow work of proving it mattered.

OpenAI's "sentiment neuron" (2017)

In April 2017, Alec Radford, Rafal Jozefowicz, and Ilya Sutskever at OpenAI published a paper called "Learning to Generate Reviews and Discovering Sentiment". They trained a network on more than 82 million Amazon product reviews with one job: predict the next character of text. It took a month on four graphics processors. Like the cat experiment, nobody labelled anything. The reviews weren't marked positive or negative.

When the researchers looked inside, they found a single unit in the network that tracked whether a review was positive or negative. It had emerged on its own, because to predict the next character of a review well, it helps a great deal to know whether the reviewer is happy or furious. Using that one unit as a feature, a simple classifier matched or beat the best existing sentiment systems on a standard test, the Stanford Sentiment Treebank, while needing far fewer labelled examples than those systems had. The researchers could even switch the unit on or off to make the network write glowing or scathing reviews.

This is a small experiment by today's standards, but it is the self-supervised idea in miniature. Train a model to predict text, and it ends up learning useful concepts that nobody asked it to learn. Scale that up by several orders of magnitude, swap the review network for a transformer, and you have the recipe behind today's language models. Radford went on to be the lead author of OpenAI's first GPT paper the following year.

An everyday scenario: the yarn shop's customer groups

Imagine Dana, who runs an online yarn shop with about 3,000 customers. She has always thought of them as two groups: beginners and experienced knitters. She exports a spreadsheet with one row per customer: total spent, number of orders, average order size, months since the first order, months since the last order, and the share of each customer's spending on yarn, patterns, and tools.

She uploads it to a chatbot that can run analysis, asks it to cluster the customers into a handful of groups and describe each one, and gets back five. Two match her beginners and experts. The other three surprise her. One group places one large order every autumn and nothing else, which she realises is people knitting holiday gifts. Another buys only patterns and almost no yarn; they're buying their yarn somewhere else. The third is a small group of big spenders who haven't ordered in eight months. None of those groups had a name in her head before, and each suggests a different action: a September gift-knitting email, a yarn bundle for the pattern buyers, a personal note to the lapsed regulars. She treats the groups as a starting point, checks a dozen real customers from each against her own memory, and drops one group that turns out to be caused by a spreadsheet error. That's unsupervised learning doing what it's good at, with a human doing what it isn't.

Try it yourself

You need a spreadsheet of your own data and a chatbot that can open files and run analysis. At the time of writing, ChatGPT, Claude, and Gemini can all read an uploaded spreadsheet and run calculations on it, but free plans limit how many files you can upload and how much analysis you can run, and these limits change often, so check your plan's current terms first. Allow an hour.

  1. Get a feel for it first (ten minutes). Open the free "Visualizing K-Means Clustering" page by Naftali Harris (linked below, no account needed). Pick a starting method and a dataset, then click through the steps and watch the markers move and the dots change colour. Try the "Smiley" dataset and notice how k-means gets it wrong. That's your first lesson: the algorithm always finds groups, even when they're the wrong ones.
  2. Choose your data. Customers, products, blog posts, or orders all work. Aim for at least a few hundred rows. Each row is one thing you want to group; each column is a fact about it. Numbers work best: amounts, counts, dates turned into "months since".
  3. Remove personal details before uploading. Delete names, emails, addresses, and phone numbers, and replace them with a plain ID number. You don't need them to find groups, and they shouldn't leave your computer without a good reason. Check what your chatbot's terms say about uploaded data; see privacy.
  4. Upload and ask for a first look. Paste this prompt with your file:
This spreadsheet has one row per [customer/product/order]. Before any analysis, describe the columns, point out missing or strange values, and tell me which columns are on very different scales.
  1. Ask for the clusters. Then paste:
Please cluster the rows into groups of similar items. Scale the numeric columns first so that no single column dominates. Try between 3 and 7 groups and tell me which number gives the clearest separation and why. For each group, give it a plain-English name, the number of rows, and the typical values that make it different from the others. Show a simple chart if you can.
  1. Check it against reality. Ask for five example IDs from each group, look them up in your own records, and ask yourself whether they really belong together. If a group looks strange, ask the chatbot which columns are driving it.
  2. Look for the oddballs. Ask:
Which 10 rows are the most unusual compared with everything else, and what makes each one unusual? Don't assume unusual means bad.

Some will be data errors worth fixing. Some might be your most interesting customers.

  1. Turn one group into one action. Pick the group that surprised you most and try one small thing aimed at it: an email, a bundle, a phone call. Then check whether it worked. The clusters are only useful if they change what you do.

Common mistakes and misunderstandings

Thinking the groups are facts. A clustering algorithm will always return groups, because that's what it's built to do. Ask k-means for five groups in completely random data and it will give you five. Whether the groups mean anything is a separate question that only checking against reality can answer.

Letting one column take over. If one column is "total spent" in dollars running into the thousands and another is "number of orders" running from 1 to 10, an unscaled algorithm will group customers almost entirely by spending, because those numbers are so much bigger. Scaling the columns first, putting them all on a comparable range, is usually the fix. That's why the prompt above asks for it.

Agonising over the "right" number of groups. There usually isn't one. Three groups might be right for a marketing plan and eight for a product review. Choose the number that gives groups you can describe and act on, and treat it as a decision, not a discovery.

Assuming unusual means bad. Anomaly detection finds things that are different, not things that are wrong. A flagged transaction may be fraud, or it may be a loyal customer buying a wedding present. Use anomalies as a list of things to look at, not a list of things to block.

Confusing unsupervised with unsupervised by humans. The word describes how the model learns, without labelled answers. It says nothing about whether people check the results. Any decision that affects real people still needs a human in the loop.

Believing no labels means no bias. Unsupervised methods learn whatever patterns are in the data, including unfair ones. If past data reflects unequal treatment, clusters will faithfully reproduce it, and a segment can end up standing in for age, gender, or neighbourhood without anyone intending it. The embeddings that learned "man is to computer programmer as woman is to homemaker" were trained without a single label. See bias.

Calling a chatbot "unsupervised". A language model's pretraining is self-supervised, and the chatbot you talk to has also been through supervised fine-tuning and training on human preferences. It learned from a mixture of all three styles. "Unsupervised" on its own gives the wrong idea.

Thinking self-supervised means no humans were involved. People still chose, filtered, and cleaned the data, designed the guessing game, and decided when training was done. What self-supervised learning removes is the labelling, not the human judgement.

Related elements