It seems like every type of work is being given its own specialised agent. There are coding agents, research agents, support agents, and sales agents, each with their own instructions and capabilities. Sometimes these agents are even given names and personalities, like human employees. When tasks cross multiple agents' responsibilities, we make them talk to each other. We might even create a "chief of staff" agent to coordinate between them.
But I'm still not convinced that this is the best approach to building with agents. We have a tendency to build structure around LLMs. We decide in advance what they should know and do, yet their power comes from adapting to the situation in front of them. Why replace deterministic workflows with generally intelligent models, only to restrict them?
Everyday work doesn't always fit neatly into role divisions anyway. Investigating an on-call issue might involve spelunking through logs, drafting customer communications, implementing a fix, and owning the pull request. Passing that work between specialist agents adds overhead while creating opportunities to lose context. Why not let the same agent follow the problem from start to end?
The primary argument for specialised agents is context control. Humans have to specialise because no one person can know everything. Agents' contexts are finite too, and irrelevant information wastes tokens while hurting performance. But agents can update their knowledge much more easily than humans, so why should limited contexts require specialised agents?
Let's imagine that an agent can work from a desk in a library. Like a human, the agent has a broad understanding of the world, but doesn't know every detail about every topic. However, the library can have books covering anything from plant genetics to how our company talks to customers. The library can hold infinite knowledge, without requiring the same of the agent.
When the agent needs to know more, it can wander through the bookshelves, choose a relevant book, and read it. The content is loaded into its context just-in-time within the session. The library can be stored in local files, synced through a shared workspace, or accessed with a cloud service. Wherever the library lives, the agent only needs to know how to find the right book.
When I advocate for "just one agent", I don't mean just one instance. We can still have multiple instances working in tandem, each with a separate context to avoid pollution. But they can be instances of the same general agent, dynamically gathering whatever knowledge and capabilities their work requires.
When building collections of specialised agents, we limit ourselves to the role definitions that make sense to us today. I'd rather have just one agent with a damn good library card.