Aiodic

Back to the table

86 Advanced Pv Privacy Safety & Ethics

Privacy

Protecting personal and sensitive data.

What it is

Privacy, in the world of AI, is about what happens to information once it leaves your hands. Every time you type into a chatbot, upload a file, connect an app, or let an assistant read your inbox, some information travels to a company's computers. Privacy is knowing what that information is, where it goes, how long it stays there, who can see it, and what it's used for, and then choosing what to share with that in mind.

It covers two kinds of data. The first is your own: your questions, your drafts, your health worries, your business plans. The second is other people's: your customers' names and emails, a client's contract, a friend's message you pasted in to ask "how should I reply to this?". The second kind is where most people get into trouble, because it isn't yours to give away, and in many countries the law says so.

To see why AI raises its own privacy questions, it helps to follow what happens to a single message you type into a chatbot. At the time of writing, with the big consumer chatbots, roughly this:

None of this is secret. It's in each company's privacy policy and help pages. But the details differ between products, between free and paid plans, and between one year and the next, so the only safe habit is to check the current settings and terms for the tools you actually use.

There's also a third, stranger privacy question that has nothing to do with what you type. AI models learn from enormous collections of text gathered from the internet and elsewhere, called datasets. That text contains personal information: names, addresses, phone numbers, things people posted years ago. Researchers have shown that models can sometimes repeat pieces of that training text word for word. So privacy in AI is also about what the model itself might have absorbed and could give back.

Why it matters

Once it's sent, you can't unsend it. A spoken secret fades. Typed text sits on a server, in backups, in logs, possibly in a training dataset, possibly in front of a reviewer. Deleting a chat removes it from your screen, but whether and when it's erased everywhere else depends on the company's rules, and occasionally on a court, as one of the examples below shows.

Other people's data comes with legal duties. If you run a business and your customers live in the European Union or the United Kingdom, their personal data is protected by the GDPR (the General Data Protection Regulation) or its UK version. Put simply, you need a good reason to use someone's personal data, you have to tell them what you do with it, you should use no more than you need, and you must keep it safe. Sending customer details to an AI service is "using" them. The most serious breaches of the GDPR can bring fines of up to €20 million or 4 percent of a company's worldwide yearly turnover, whichever is higher, although regulators usually start with warnings for small firms. Many US states and other countries have their own privacy laws with similar ideas.

Confidential isn't only personal. A client's unreleased book, a supplier's price list, your own source code or business plan: none of this is "personal data" in the legal sense, but it's confidential, and pasting it into a consumer chatbot may break a contract or a promise. The first example below is about exactly that.

Trust is a business asset. Customers increasingly ask what you do with their data and whether you use AI. A clear answer builds trust. A leak, or the discovery that their private messages were pasted into a chatbot without their knowledge, loses it quickly. Governments have learned this the hard way too: the Dutch tax authority's childcare benefits scandal, covered in Human in the Loop, ended with a €2.75 million fine from the Dutch data protection authority for how it had used people's personal data.

AI tools collect more than older software did. A spreadsheet only knows what's in the spreadsheet. An assistant that can read your email, browse the web as you, or watch your screen has access to far more. Microsoft's Recall feature, covered in Memory, showed how quickly "helpful" can turn into "a searchable record of everything". The more an AI tool can do, the more carefully you need to decide what it's allowed to see.

Leaks travel in new ways. Traditional leaks come from hacks. With AI, data can also escape through a shared link, a planted instruction that tricks an assistant into sending information out, a forgotten connection to another app, or a model repeating something from its training. Knowing these routes is most of the defence.

Real-world examples

Samsung engineers paste secrets into ChatGPT (March to May 2023)

On 11 March 2023, Samsung's semiconductor business lifted an internal ban on using ChatGPT, to help staff work faster. Within about three weeks, according to Korean media reports summarised by The Register on 6 April 2023, there had been at least three incidents. One employee pasted the source code of a faulty semiconductor database program into ChatGPT and asked for a fix. Another uploaded code designed to identify defective equipment. A third turned a recording of a meeting into a transcript and asked ChatGPT to write the minutes.

None of this was a hack. The employees were doing what the tool was sold for. The problem was where the data went: at the time, ChatGPT's consumer service could use conversations to improve its models unless users opted out, and OpenAI's own FAQ said conversations might be reviewed by its AI trainers. Samsung reportedly responded with emergency measures, including limiting each question to 1,024 bytes of text.

On 2 May 2023, Bloomberg reported that Samsung had told staff in one of its biggest divisions, in a memo, that they could no longer use generative AI tools such as ChatGPT on company devices and networks. The memo said the company was concerned that data sent to these services was stored on outside servers, was hard to retrieve and delete, and could end up being shown to other users. An internal survey the month before had found that 65 percent of respondents saw such tools as a security risk.

The lesson for a small business is simple. Your team will use AI whether or not you have a policy. A short, clear rule about what may and may not be pasted in, plus a business plan that doesn't train on your data, prevents the Samsung problem far better than a ban nobody follows.

A bug, a ban and a fine: ChatGPT in Italy (2023 to 2026)

On 20 March 2023, some ChatGPT users noticed they could see the titles of other people's conversations in their sidebar. OpenAI took ChatGPT offline and, on 24 March, published an explanation. A bug in an open-source software library it used, called redis-py, had caused some users to receive data meant for others. Besides chat titles, the first message of a new conversation could sometimes appear in someone else's history. And for about 1.2 percent of ChatGPT Plus subscribers who were active during a nine-hour window that day, another user's name, email address, payment address, card expiry date and the last four digits of their card number may have been visible. Full card numbers weren't exposed. OpenAI contacted the affected users.

Italy's data protection authority, the Garante, acted fast. On 31 March 2023 it ordered an immediate temporary limit on OpenAI processing the data of people in Italy, citing the breach, the lack of a clear legal basis for using people's data to train the models, and the lack of any age check for children. OpenAI made ChatGPT unavailable in Italy. After OpenAI added clearer privacy notices, a way for Europeans to object to their data being used for training, and an age check, the Garante announced on 28 April 2023 that ChatGPT could return.

The investigation continued. In December 2024 the Garante announced a €15 million fine, finding among other things that OpenAI hadn't notified it of the March 2023 breach and had used personal data to train ChatGPT without an adequate legal basis or proper information to users. It also ordered a six-month public awareness campaign in Italian media. OpenAI called the fine disproportionate and appealed. On 18 March 2026, the Court of Rome annulled the fine and the campaign. According to legal commentators, the court's reasoning, published in May 2026, didn't rule on whether OpenAI had broken the law. It held that once Ireland's regulator recognised OpenAI's Irish company as its main European base in February 2024, the Italian authority no longer had the power to decide the case. At the time of writing, a further appeal remains possible.

Two lessons stand out. Even a well-funded AI company can leak one user's data to another through an ordinary software bug. And the question of whether AI companies may train on personal data at all is still being argued in courts and regulators' offices, which is one reason the terms keep changing.

Models that remember their training data (2021 and 2023)

In 2021, Nicholas Carlini and a team of researchers from Google, OpenAI, Stanford, Berkeley and other institutions presented a paper at the USENIX Security Symposium called "Extracting Training Data from Large Language Models". Using only the ability to ask GPT-2 questions, they recovered hundreds of passages it had memorised word for word from its training data, including names, phone numbers, email and postal addresses of real people, as well as source code.

Many people assumed that the polished chatbots built later would be safe from this. On 28 November 2023, a team led by Milad Nasr and Nicholas Carlini, mostly from Google DeepMind, showed otherwise. They asked ChatGPT to repeat a single word, such as "poem", forever. After repeating it several hundred times, the model sometimes drifted off and began producing other text, and some of that text was copied directly from its training data. Spending about $200 on queries, they extracted more than 10,000 unique memorised examples. In one test, 16.9 percent of the outputs they checked contained memorised personal information, such as email addresses and phone numbers. They told OpenAI about the flaw on 30 August 2023 and waited the standard 90 days before publishing.

That particular trick was specific to one model and was dealt with. The underlying point wasn't. As the researchers put it in their write-up, patching one exploit isn't the same as fixing the underlying problem: models can memorise parts of what they were trained on. For you, this has two practical consequences. Anything you allow to be used for training could, in principle, be learned by a future model. And if you build or fine-tune a model on customer records, the model itself can become a copy of that data.

Shared chats turn up in Google (2025 and 2026)

Most chatbots let you share a conversation by creating a link. In 2025, ChatGPT's share menu included an extra checkbox to make the chat discoverable by search engines. At the end of July 2025, Fast Company reported that a simple Google search for ChatGPT's shared-link address turned up nearly 4,500 conversations, some describing addiction, trauma and mental health struggles in enough detail to identify the people involved. On 31 July, OpenAI's chief information security officer, Dane Stuckey, announced that the company was removing the option, which he called a short-lived experiment, because it gave people too many chances to share things they didn't intend to. OpenAI said it was working to remove the indexed chats from search engines.

A year later it happened again with a different company and a different cause. On 25 July 2026, a Reddit user showed that searching Google for Claude's shared-link address surfaced many shared Claude conversations and Artifacts, the small documents and apps people build inside Claude. TechCrunch reported on 27 July 2026 that some reportedly contained health records, internal company documents, and the names and phone numbers of children. Anthropic said it doesn't give search engines lists of shared chats, and that shared links only appear in search if they've been posted somewhere public, such as a forum or social media. By that afternoon, TechCrunch's test search no longer returned results.

The lesson is the same for every product: a shared link is a public document. Anyone who has the link can read it, and links get forwarded, pasted and indexed. Before you share, read the conversation as if a stranger were reading it.

A court orders chat logs to be kept (2025 to 2026)

In the copyright lawsuit brought by The New York Times against OpenAI, the newspaper argued that evidence might be in users' chats. On 13 May 2025, Magistrate Judge Ona Wang ordered OpenAI to preserve output logs that would normally be deleted, including chats users had deleted themselves. The order covered ChatGPT Free, Plus, Pro and Team and ordinary API use, but not ChatGPT Enterprise, ChatGPT Edu, or API customers with zero data retention agreements. OpenAI publicly objected, saying the order conflicted with its privacy commitments.

On 9 October 2025, the judge ended the obligation to keep new logs from 26 September 2025, though logs already preserved, and accounts flagged by the Times, stayed covered. Separately, the court ordered OpenAI to hand over a sample of 20 million de-identified ChatGPT logs, and on 5 January 2026 District Judge Sidney Stein upheld that order. At the time of writing, the case is continuing.

For ordinary users, this is a reminder that "deleted after 30 days" is a company policy, not a law of nature. A court order, a legal dispute or a policy change can keep your words for longer than you expected. The safest data is the data you never sent.

An everyday scenario: the bookkeeper's shortcut

This is an illustrative scenario, not a documented event. Imagine Priya, who does the books for a dozen local shops. Late one evening, tired, she exports a client's customer list with names, emails and purchase totals and pastes it into her free chatbot account to ask for a summary of the best customers. It works beautifully.

The next morning she remembers that her account still allows conversations to be used for training, that she never told the client she uses AI, and that her contract with them says their data stays with her. Nothing bad has happened yet, and probably nothing will. But she can't take the data back. She deletes the chat, switches her account's training setting off, and from then on removes names and emails before asking the chatbot anything, replacing them with "Customer 1", "Customer 2" and so on. The summary is just as useful without them.

Try it yourself

This takes about 15 minutes per AI account. Do it for each chatbot you use regularly. Menus move often, so if you can't find something, search the product's help pages for "data controls", "privacy" or "training".

  1. Find the training setting. At the time of writing, ChatGPT has a switch called "Improve the model for everyone" under Settings, then Data Controls. Claude has a model training choice in its Privacy settings; Anthropic changed its consumer terms in September 2025 so that Free, Pro and Max users choose whether their chats are used for training, with chats kept for up to five years if they say yes. Gemini ties review and improvement to a setting called Keep Activity. Decide deliberately whether you want it on or off, and check what your plan does by default.
  2. Check your memory. Open the memory or personalisation settings and read what's saved. Delete anything too personal, out of date, or about other people. The Memory article walks through this in detail.
  3. Review your shared links. Most chatbots have a list of links you've shared, usually in the data controls or sharing settings. Delete any you no longer need, especially old ones.
  4. Check connected apps. Look for a section called connectors, apps, integrations or extensions. Disconnect anything you don't use. Every connection to your email, calendar or cloud drive is a door into your data.
  5. Learn the temporary chat option. Most chatbots offer a temporary or incognito mode that isn't saved to your history or memory and, at the time of writing, isn't used for training. It's still kept for a short time for safety checks, so it's more private, not invisible.
  6. Ask the chatbot to explain its own policy, then verify. Paste this in, then check the answer against the real help pages it points to, because chatbots can be out of date about their own products:
Explain in plain language what happens to what I type here.
Cover: whether it can be used to train models, how long it's
kept, whether people at your company can read it, and what
changes if I use a temporary chat. Tell me which help or policy
page I should read to confirm each answer.
  1. Adopt a "safe to paste?" check. Before pasting anything, ask yourself these questions. If any answer is yes, stop and remove or replace the sensitive parts first.
  1. Practise removing personal details. When you do need help with real material, strip it first. You can even ask the chatbot to help you build the habit, using made-up text:
Here is a made-up customer email. Rewrite it with every name,
email address, phone number, street address and order number
replaced by a placeholder like [NAME 1] or [EMAIL 1]. Then list
any other details that could still identify the person.

"Hi, it's Maria Lopez from 14 Elm Street. My order 55812 never
arrived and you can reach me on 555 0142 or [email protected]."

Then use placeholders like these in your real work. The chatbot doesn't need to know who Maria is to help you write a polite reply.

  1. If you run a business, write down three things. Which AI tools your business uses, what customer data may go into them, and which plan each one is on. Then check that your privacy notice tells customers you use these services, and read each company's business terms or data processing agreement. The European Data Protection Board's guide for small businesses, listed below, is a good free starting point.

Common mistakes and misunderstandings

"Deleting the chat deletes the data." It removes it from your view. At the time of writing, the major chatbots erase deleted chats from their systems after a period, often around 30 days, but legal holds, backups and training that has already happened can keep copies longer. Treat deletion as tidying up, not as a guarantee.

"Paid means private." Not always. A paid personal plan such as a monthly subscription is often covered by the same consumer terms as the free plan, including training settings. The stronger promises usually come with business or team plans and API access. Check which terms your plan falls under.

"Business plans mean I can paste anything." Business terms usually mean your data isn't used for training and is handled under a contract. That's a big improvement, but the data still leaves your control, and you still need a lawful reason to use customer data in the first place and to tell customers about it.

"Anonymised means I removed the name." A description can identify someone without a name: "the only vet in our village who had a complaint last March". Remove details in combination, not just the obvious ones, and use the least data that gets the job done.

Pasting secrets for a quick fix. Passwords, API keys and login codes should never go into a chatbot, even to ask "why isn't this working?". Replace them with a placeholder. The API article describes what can happen when a key leaks.

Forgetting the rest of the chain. A chatbot is rarely the only company involved. An automation that sends customer emails to an AI step passes data through your email provider, the automation tool, the AI company and wherever the results go. See Workflow Automation for how to send each step only the fields it needs. The same applies to uploaded spreadsheets, as described in Data Analysis.

Trusting the chatbot's description of its own settings. Models often describe old versions of their own products or make things up, a problem known as hallucination. Always confirm with the company's current help pages.

Treating shared links as private. "Anyone with the link" means anyone. Share a copy with the private parts removed, or share a screenshot of just the part you need.

Letting agents roam without limits. Browser agents and other assistants that act for you can read whatever they can reach. Give them only the accounts and folders the task needs, and watch for hidden instructions in web pages or documents that try to make them send your data somewhere else.

Related elements