Before "agentic AI" was on every conference banner, I was using an early version of CrewAI to get an LLM to read, search and document decades-old COBOL for a financial services company in Paris. Here is what an AI agent actually is, how it worked back then, and why the idea is suddenly everywhere.
This year it is hard to open a newsletter without meeting the word "agentic". Agentic AI, agentic workflows, agentic platforms, agentic everything. Every software company now has agents, and every conference has a keynote about them. If you're new to all this, it can sound as though a brand-new species of AI arrived some time in 2025.
It didn't. The core idea is older, and much simpler, than the marketing suggests. I know because I built with it before it had a name. In the early days of CrewAI, one of the first frameworks for building teams of AI agents, I used it on a project for a financial services company in Paris: getting an LLM to translate and document COBOL, the programming language a surprising amount of the financial world still quietly runs on. To do that, the model had to go and look things up for itself. That, it turns out, is the whole trick.
This piece explains, in plain terms, what an AI agent is, how that early version worked, a practical example you can play with, and how it all connects to the agent boom happening now. No technical background needed. If you'd like the foundations first, there's What is AI, really? and the piece on how LLMs work, but you don't need either to follow along.
COBOL was designed in 1959, and a remarkable share of the world's banking, insurance and payments still runs on it. These systems work. Many have worked for decades. The trouble is that the people who wrote them have often retired, the documentation (where it exists) tends to describe the system as it was long ago, and the code itself is enormous, tightly interconnected, and written in a style few modern developers are trained in.
The job was twofold: translate COBOL programs into a modern language, and document what they actually do, in plain words a new team could rely on. An LLM looks perfect for this. It can read code and it can write explanations. And on a single, self-contained program, it does well.
But real COBOL programs are rarely self-contained. A program uses data structures defined somewhere else (in shared files called copybooks), calls other programs, and reads files that yet other programs wrote. To explain what one program does, you often need to know about a dozen others. The model had never seen any of them. It knew COBOL the language. It knew nothing about this codebase.
You couldn't just paste the whole codebase in, either. The amount of text an LLM could take in at once, its context window, was far smaller then than it is today, and a financial institution's codebase is far larger than any window. So the model needed a way to look up exactly the pieces it needed, when it needed them. The codebase went into a vector database, a searchable index of the programs and copybooks, and the model was given the ability to query it.
That last sentence is the whole idea of an AI agent.
Strip away the marketing and an agent, at least as it was then, is three things: an LLM, some tools, and a loop.
Think of a very well-read consultant locked in a room with no phone, no internet and no files. Ask them anything and they'll answer from memory: often impressively, sometimes confidently wrong. That's a plain LLM. Now give the same consultant a phone, a library card and the key to your filing cabinet, and tell them they're allowed to use all three before answering. That's an agent. Same brain. Very different results.
One detail surprises people. The LLM can't actually press any buttons. It can only write. So when it wants to use a tool, it writes a request in a fixed format, something like search_codebase("CUST-BALANCE"). The framework around it (ordinary code, which CrewAI provided) spots the request, runs the search, and pastes the result back into the conversation. The model reads the result and decides what to do next: another search, a different tool, or a final answer.
That cycle (think, act, look at the result, repeat) has a name in the research literature: ReAct, short for "reasoning and acting", from a 2022 paper. Nearly every agent you'll hear about today, however sophisticated, still runs some version of it. Step through one below. The COBOL is invented, but the shape of the problem is real.
Look at the two answers side by side. The plain LLM's answer sounds perfectly reasonable, and that's the danger: it guessed that the rate was monthly, and it was wrong. It also missed the leap-year issue entirely, because spotting that required going and checking. The agent's answer is less fluent and far more useful, because every claim in it came from somewhere.
CrewAI appeared as an open-source project in late 2023, and its central idea was charmingly human. Instead of programming an agent, you describe it, much like writing a job posting.
Each agent gets a role, a goal and a backstory, plus the tools it's allowed to use. You then write tasks, each with a description and what a good result should look like, and assign them to agents. Put the agents and tasks together and you have a crew. Finally you choose a process: sequential, where tasks run one after another and each passes its output to the next, or hierarchical, where a manager agent hands out the work and checks it.
The backstory sounds like a gimmick. It isn't quite. LLMs respond strongly to framing, so an agent told it has spent thirty years maintaining banking systems tends to write in a more careful, more domain-aware way than one told nothing at all. You are, in effect, prompting with a character. Here is a simplified version of how you might set up a crew for a job like the COBOL one.
analyst = Agent( role="COBOL analyst", goal="Work out exactly what each program does", backstory="Thirty years maintaining banking systems", tools=[codebase_search], ) document = Task( description="Document what program INTCALC does", expected_output="A plain-English summary, with sources", agent=analyst, ) crew = Crew(agents=[analyst, engineer, writer], tasks=[document, translate, write_up], process=Process.sequential) crew.kickoff() # ...and off it goes
There was no established playbook for any of this yet. But of everything involved, the part that was genuinely, stubbornly hard was knowledge: making sure that when the model searched, it found the right thing, and that the right thing was enough.
COBOL makes this harder than most. Names tend to be short and cryptic (CUST-BAL, WS-AMT-2), the same idea can be called different things in different programs, and the meaning of a program can depend on a copybook three references away. A search that works well on ordinary documents can return a dozen plausible-looking pieces of code and miss the one that matters. When that happens, the agent doesn't know it's missing something. It reasons confidently from whatever it was given.
This is the lesson every agent project since has rediscovered. The model can only reason about what's in front of it. Deciding what goes in front of it is most of the work. The demo below shows the trade-off. Each square is a file in a codebase. Six of them (the gold outlines) are the ones the agent actually needs. The shaded area is what fits in its context window.
Today's models have far bigger context windows, and that helps. But notice what the large window does: it gets everything in by also letting in a crowd of things that don't matter, and models get distracted by irrelevant material just as people do. Search gets the essentials into a small window, except for the ones it misses, which the agent never knows about. Every serious agent project ends up spending much of its effort here.
The PAD Economic Analysis tool on this site, built much more recently, is a team of seven specialised agents that search a Project Appraisal Document through a vector database before they do any economics. A different decade of documents, the same shape, and the same hard part.
"The model can only reason about what's in front of it. Deciding what goes in front of it is most of the work."
COBOL is a niche passion, so here's a problem anyone can picture. You ask an assistant: "Book my trip to the workshop in Niamey on 12 November, and keep it under €1,200."
A plain chatbot can't do this. It can only tell you how you might do it. An agent can, if it has the right tools: a calendar to check you're free, flight search, a currency converter (prices in Niamey are in CFA francs), your organisation's travel policy, and email to send the booking request. Choose which tools this agent gets, and whether it must ask you before acting, and watch how its plan changes.
Two lessons fall out of this. First, an agent is only as capable as its tools. Take away flight search and it doesn't stop; it guesses, and its guesses sound exactly as confident as its facts. Second, every tool that makes it more capable also gives it more ways to do something you didn't intend. The email tool is what turns a helpful plan into an action in the world. That's why the "ask me first" switch exists, and why every serious agent product today has some version of it.
When I was using CrewAI, agents were a niche: a handful of open-source frameworks, an enthusiastic community, and a lot of experiments that worked brilliantly in demos and fell over in practice. Today they're the main story in AI. Here's roughly how we got from there to here.
Look at that list and what changed is mostly not the idea. It's the plumbing, and the models. Models are now trained specifically to use tools well, so they pick the right one more often and format their requests reliably. Context windows grew enormously. Standards like the Model Context Protocol mean a tool built once can be plugged into many different AI systems. And models got good enough to stay on track through long, multi-step jobs: writing and testing code, researching a question across dozens of sources, filling in forms on a website.
What hasn't changed is the core. An LLM, some tools, a loop. If you followed the COBOL agent above, you understand the coding agent that writes software and the research agent that reads fifty web pages for you. They're the same machine, with better parts and a much longer leash.
"An agent is an LLM, some tools and a loop. Everything since has been better parts and a longer leash."
Knowledge is still the hard part. Bigger windows and better search help, but an agent that silently misses the one document that matters will still reason confidently without it.
Small errors add up. Every step in the loop is the model making a decision: which tool, which search, whether it's done. Suppose each step goes right 95% of the time. That sounds excellent. But a job that takes 20 steps only goes right from start to finish if every step does, and the odds of that are much worse than you'd think. Try it.
Acting is riskier than answering. A wrong answer can be ignored. A sent email, a submitted form or a deleted file cannot. And agents read whatever they find, including web pages or documents written by someone hoping to steer them. More tools, more reach, more ways for that to matter.
Every step costs time and money. Each turn of the loop is another call to the model. An agent that searches twenty times to answer one question is twenty times more expensive than a chatbot that answers from memory. Sometimes that's worth it. Often a simpler design would do.
Notice that most of these come back to the same thing: at every step, a model built to write is being asked to make a small decision. Which tool? Is this result good enough? Am I done? That turns out to be a problem worth a piece of its own, and it's the subject of next month's essay on Jev, a model built to make exactly those decisions.
Before it was a thing, an agent was just a sensible way to let an LLM look things up. It still is. The COBOL project needed a model that could go and find the copybook it was missing, and that is all an agent really is: a model that's allowed to go and find out, and then do something with what it found. The word got bigger. The idea stayed the same size.
So if someone pitches you an "agentic" product, you don't need to understand the keynote. Ask four questions. What tools does it have? What knowledge can it reach? How does it decide when it's done? And what is it allowed to do without asking you first? The answers will tell you more than any demo.
Think, act, look, repeat. Everything else is detail.