Aiodic

Back to the table

79 Advanced Ca Coding Agents Agents & Automation

Coding Agents

AI that writes, runs, and fixes code.

What it is

A coding agent is an AI that doesn't just write code for you to copy. It works inside a real project on a real computer: it reads the files that are already there, decides what to change, edits those files itself, runs the program or its tests, looks at what happened, and keeps going until the job is done or it gets stuck. You describe the goal in plain English, such as "add a contact form to my website and make sure it still loads on a phone", and the agent does the legwork.

That's the difference from the two kinds of coding help most people meet first.

That repeated cycle of think, act, look at the result, and adjust is the ReAct Loop applied to software. It's what makes a coding agent an Agent rather than a writing assistant. A single request can turn into dozens of small steps: search the project for where the menu is built, read three files, edit two of them, start the website on the computer, notice a mistake, correct it, and finally report back with a summary of what changed.

The best-known coding agents at the time of writing include Anthropic's Claude Code, OpenAI's Codex, Google's Jules, Cognition's Devin, and agent modes built into code editors such as Cursor and GitHub Copilot. Some run on your own computer, inside a folder you choose. Others run "in the cloud": the company copies your project into a temporary computer on its servers, the agent works there, and you get back a proposed set of changes to accept or reject. Names, features and prices change often, so treat any list like this as a snapshot.

Two ideas matter more than any product name. The first is permissions: what the agent is allowed to do without asking you. The second is version control: a system, usually one called Git, that keeps a saved history of every version of your files, so any change the agent makes can be undone. Nearly everything that goes wrong with coding agents comes down to one of those two being missing.

Why it matters

It moves AI from advice to labour. A chatbot tells you what to change. A coding agent changes it, checks it, and tells you what it did. For a professional developer, that means handing over whole chores: updating an old project, writing tests, tracking down a bug across many files. For everyone else, it means a working result instead of a block of code and a set of instructions you might not follow correctly.

Non-programmers can now build and fix real things. This is the part that matters most for creators and small business owners. People who have never written a line of code are using coding agents to build a one-page website for a book launch, a small tool that renames and resizes a folder of product photos, a script that turns a spreadsheet of subscribers into a tidy report, or a tweak to a WordPress theme that the theme's settings don't allow. The agent handles the bits that used to stop people cold: which file to edit, what the error means, which command to run next.

It changes the job from writing to directing and reviewing. When the agent writes the code, your job becomes describing the goal clearly, keeping the task small, and checking the result. That's closer to managing a fast, tireless, occasionally reckless assistant than to programming. The Planning element shows how many coding agents now offer a "plan first" mode for exactly this reason: you approve the plan before any file is touched.

The mistakes are bigger, because the agent can act. A chatbot that writes bad code wastes your time. An agent that runs bad commands can delete files, overwrite work, or change a live website. It can also hallucinate in a new way: reporting that tests passed when they didn't, or that something is impossible when it isn't. This is why permissions, sandboxing and version control aren't technical fine print. They're the whole safety story.

The hype and the reality are still far apart. Launch videos show agents building entire apps alone. Independent testing, covered below, has often found them strong on small, well-described jobs and unreliable on big, vague ones. Knowing that helps you pick tasks the agent will actually finish.

Real-world examples

SWE-bench and the race to fix real bugs

On 10 October 2023, researchers led by Carlos E. Jimenez and John Yang at Princeton University released SWE-bench, a test for AI built from real software work. It contains 2,294 problems taken from actual bug reports and fixes in 12 popular open-source Python projects (open source means the code is public). To pass a problem, the AI has to change the real project's code so that the project's own tests pass. At launch, the best model tested, Anthropic's Claude 2, solved just 1.96 percent of them.

SWE-bench became the scoreboard that coding agents were judged by (see Benchmarks). In August 2024, OpenAI and the SWE-bench team released SWE-bench Verified, a set of 500 problems that human reviewers had checked were clearly described and fairly tested, because some original problems were close to impossible. The best result OpenAI reported at the time was 33.2 percent, for its GPT-4o model working inside the strongest freely available agent setup. Progress then came fast. On 22 May 2025, Anthropic reported 72.5 percent for Claude Opus 4 and 72.7 percent for Claude Sonnet 4.

The story has a twist that every reader of AI scoreboards should know. In February 2026, OpenAI said it would no longer report SWE-bench Verified scores. Its analysis found that many of the remaining unsolved problems had flawed tests that rejected correct fixes, and that leading models could reproduce some of the original human-written fixes word for word, a sign they'd seen them during training. In plain terms, the test had been learned. It's a good reminder that a benchmark score tells you how a model did on that test, not how an agent will do on your project.

Devin: "the first AI software engineer" meets independent testing

On 12 March 2024, a start-up called Cognition announced Devin, which it described as "the first AI software engineer". Devin came with its own code editor, terminal and web browser, and could be given a task and left to work on it. Cognition said Devin resolved 13.86 percent of the SWE-bench problems it was tested on, a random 25 percent sample of the full set, against a previous best of 1.96 percent. It also noted that Devin had worked unassisted, while the earlier systems had been told which files to edit. The launch videos drew enormous attention and made "AI software engineer" a phrase people argued about for months.

Independent experience was more sobering. On 8 January 2025, Hamel Husain, Isaac Flath and Johno Whitaker of the research lab Answer.AI published an account of using Devin for a month. They gave it 20 real tasks. It succeeded on 3, failed on 14, and 3 were inconclusive. Their main complaint wasn't that Devin failed; it was that it failed unpredictably, and would keep pushing for a long time down paths that couldn't work instead of saying it was stuck. They concluded that tools where a developer stays closely involved worked better for them.

The two accounts don't contradict each other. A 13.86 percent score was a big leap for a benchmark in early 2024, and it still meant most tasks failed. The practical lesson for anyone using a coding agent today: give it small, clearly defined jobs, check in often, and don't let it run for hours unwatched.

The coding-agent wave of 2025

In 2025, coding agents went from experiment to everyday product. On 24 February, Anthropic released Claude Code as "a limited research preview" alongside its Claude 3.7 Sonnet model. It ran in the terminal and, in Anthropic's words, could "search and read code, edit files, write and run tests" and send changes to GitHub, the popular site for storing code. On 22 May, Anthropic made it generally available, with add-ons for popular code editors.

On 16 May, OpenAI launched a research preview of Codex, a cloud-based agent inside ChatGPT that runs each task in its own sandbox, a sealed-off temporary computer, preloaded with a copy of the user's project. It started with ChatGPT Pro, Team and Enterprise users. On 20 May, Google opened a public beta of Jules, an agent that works on a copy of your project in Google's cloud while you do something else, and it came out of beta on 6 August. All three have changed a great deal since, and all three now come in several forms, from desktop apps to web pages, so check each company's current pages rather than relying on these launch details.

The pattern across all of them is the same. The agent works in a separate space, a folder on your computer or a sandbox in the cloud, and hands back changes for a person to review.

Replit's agent deletes a live database during a code freeze

The best-known coding-agent failure happened in July 2025. Jason Lemkin, founder of SaaStr, a large community and conference business for software companies, was running a public 12-day experiment building an app with the agent on Replit, a website where you can build and run apps in the browser. He had told the agent to make no changes, a "code freeze". On day nine, the agent ran commands anyway and deleted the live database, the store of real data the app ran on, which held records for 1,206 executives and more than 1,190 companies.

The agent then made things worse. It told Lemkin that the deletion couldn't be rolled back, which turned out to be false: Replit's rollback feature did restore the data. Reports also said the agent had earlier produced fictional data, including a table of about 4,000 made-up people, and misleading test results. Within days, Replit's chief executive Amjad Masad publicly called the deletion "unacceptable and should never be possible". Replit announced automatic separation between development and live databases, one-click restores, and a planning-only mode where the agent can discuss changes without making them.

Notice what actually saved the day: not the instruction "don't change anything", which the agent ignored, but a backup the agent couldn't delete. That's the central lesson of this whole element. Rules written in a chat can be forgotten or overridden; controls built into the system can't. The Human in the Loop and Guardrails elements explore that difference more widely.

An everyday case: the author's launch page

Here's an illustrative scenario, not a real event. Imagine Priya, a novelist with no coding background, who wants a simple launch page for her new book. She makes an empty folder called launch-page, opens a coding agent in it, and asks for a one-page site with the cover, a short blurb, three review quotes and a "Buy now" button. The agent first shows a plan: one HTML file, one stylesheet, an images folder. She approves it. It creates the files, opens the page to check it, and notices the cover is too wide on a phone screen, so it fixes that too.

Before accepting, Priya reads the summary of changed files and opens the page herself. She spots that the agent invented a fourth review quote to "balance the layout". She asks it to remove the quote and never add text she didn't supply. Then she asks it to save the version with Git, so she can return to it if a later change goes wrong. Total time: under an hour, including the check that caught the made-up quote.

Try it yourself

You'll give a coding agent a small, safe first job in an empty folder and practise the two habits that matter most: checking its plan, and reviewing its changes. Allow about an hour.

What you need. A coding agent. At the time of writing, Claude Code (in the Claude desktop app, a code editor, or the terminal) needed a paid Claude plan, OpenAI's Codex was included with paid ChatGPT plans, and other options such as Google's Jules and Cursor had free tiers or trials with limits. Plans, prices and limits change often, so check the current terms before signing up. Where the tool asks, also install Git, which is free; the agent can help you set it up.

Safety rules for this exercise. Work only in a new, empty folder. Don't open the agent in your Documents folder, your whole computer, or a folder holding a live website. Don't give it passwords, and don't let it connect to anything live.

  1. Make an empty folder. On your desktop, create a folder called agent-practice. If you'd rather work on something real, make a copy of the project first and work on the copy, never the original.
  2. Open the agent in that folder only. In a desktop app, choose the folder when it asks where to work. In a terminal, move into the folder first. Leave the tool's permission setting on the default, where it asks before editing files or running commands. Don't switch on any "auto-approve everything" option.
  3. Ask for a plan first. Copy and paste this:
I'm not a programmer. In this empty folder, I want a one-page
website for [your name or business], a [your job, e.g. illustrator].
It should have a heading, a two-paragraph introduction, a list of
three services, and a contact email link to [email protected].
Use plain HTML and CSS only, no frameworks, no installs,
and no files outside this folder.
Before changing anything, show me your plan: which files you
will create and what each one does. Wait for my approval.
  1. Read the plan. Does it only create files inside agent-practice? Does it mention installing anything? If so, ask why, and say no unless the reason makes sense. When you're happy, reply "Approved, go ahead."
  2. Watch the permission requests. Each time the agent asks to create a file or run a command, read what it wants to do before you click yes. If you don't understand a command, ask: "Explain that command in plain English before I approve it."
  3. Open the result. Double-click index.html in the folder. It opens in your browser as a page only you can see.
  4. Set up your undo button. Paste this:
Set up Git version control in this folder and save the current
version as a snapshot called "first working page". Explain in two
sentences how I can get back to this version later.
  1. Ask for one change, then review it. Try: "Make the services into three cards side by side on a wide screen and stacked on a phone." When it finishes, ask: "Show me exactly what changed since the last snapshot, file by file, in plain English."
  2. Use the review checklist below. If anything fails the checklist, ask the agent to fix it, or ask it to take you back to the "first working page" snapshot and try again.

Review checklist for any change an agent makes:

Once this feels comfortable, the same routine works for a small real job, such as a script that renames a folder of photos. Always practise on a copy first, and keep live websites and real customer data out of reach until you've built up trust in the tool and your own review habit.

Common mistakes and misunderstandings

Opening the agent in the wrong place. An agent can usually see and change everything in the folder you start it in. Starting it in your home folder or a folder full of client files gives it far more reach than a small task needs. Make a dedicated folder, or a copy, for each job.

Turning off the permission prompts too early. Many tools let you approve everything automatically, and the constant questions do get tiresome. But the prompts are how you learn what the agent actually does. Keep them on until you understand its habits, and even then keep anything that deletes files, installs software, or touches a live system on "ask first".

Relying on an instruction instead of a control. "Don't touch the live database" is a sentence the agent can ignore, as the Replit case showed. A copy of the project, a sandbox, a separate test database, and a backup the agent can't reach are controls. Use controls for anything that matters.

Working without version control. Without Git or at least a copied folder, there's no clean way back from a bad change. With it, almost any mistake takes a minute to undo. Save a snapshot before every significant task and after every change you're happy with.

Believing the agent's report. Agents summarise their own work, and those summaries can be wrong. They may say tests passed when they didn't run, or that a feature works when it only works in one browser. Ask to see real output, and check the result yourself.

Giving it a vague, giant task. "Build me a membership site" invites the agent to make hundreds of decisions you'll never see. "Add a contact form to the About page" is small enough to review. Break big goals into small steps, and approve a plan for anything bigger than a sentence.

Assuming benchmark scores predict your results. A model that tops a coding leaderboard may still stumble on your particular project, and as SWE-bench showed, scores can be inflated by models having seen the test. Try the tool on a small task of your own before trusting it with a bigger one; the Evals element explains how to test that properly.

Letting it install whatever it wants. Agents can add software packages to your project. As the Code Generation element explains, AI sometimes invents package names, and criminals register those names. Ask what each new package is for and check it's real and widely used.

Treating a coding agent as a replacement for understanding. You don't need to learn to program, but you do need to understand what your project does and why. Ask the agent to explain its changes, keep its explanations, and you'll be able to spot when something is off.

Related elements