LLMs aren't brains.

Specialist agents

Databases

LLM

Tools

The human

asks, then reads

At my brother's graduation last year, the family conversation turned to AI. The general feeling was distrust. They've seen the headlines about lawyers citing fake cases and chatbots inventing medical facts, and they think the whole technology is overhyped.

They're not wrong about the headlines. But I think they're wrong about what those headlines mean. The mistake most people make about AI, my family included, is treating an LLM like a brain. It isn't a brain. It's the thing that connects the parts of a brain together and tells you the story.

An LLM on its own doesn't know much about the world. It can write fluently, summarize things, and translate jargon into plain language. What it doesn't do is store the actual facts. The facts have to come from somewhere else: a database, a tool, a specialist who actually knows. The LLM is what pulls those pieces together and turns them into something a human can read.

Once you see it that way, the architecture writes itself.

Patient

Router agent

triage and refer

Gastro

database

Neuro

database

Psych

database

Bones

database

Take an emergency room.

If you ask an LLM to diagnose a patient on its own, it will hallucinate. It will produce a fluent, confident answer that may have nothing to do with the patient's actual condition. But give it a database of every bone in the human body, and it can identify a fractured toe from a fractured foot. That's a better diagnosis than it could manage on its own.

But an LLM with only a database of bones won't save every patient. You'll need databases on gastroenterology, neuroscience, psychiatry, every specialty. And immediately we run into the same problem: that's too much information. Even for a human doctor. Which is why hospitals don't have one doctor who knows everything. They have specialists, each of whom has spent years inside one database.

So LLMs need to be broken into agents. One for gastro, one for neuro, one for psych. But here's the constraint. When a patient walks into a hospital, they shouldn't have to tell their story to five different doctors. There's usually one front-line doctor with a rough sense of common cases, who handles what they can and refers out, or pulls in a specialist available in the ER. We need that agent too. A router agent with high-level knowledge of every specialist available, who directs the problem to the right one and catches the specialist up on the hard truths of the patient's case.

That's the architecture I keep coming back to.

LLM

proposes an answer

Challenger

disbelieves, demands evidence

Stronger,
cited answer

It's annoying when LLMs don't come up with the right idea on the spot. Well, humans usually don't either. We work in teams, where we put our minds together and brainstorm. There might be a two-hour back-and-forth between teammates, a battle of persuading each other, pulling out all the stops to convince the other person. Sometimes the battle gets too heated. My friend at TikTok told me how her two tech leads couldn't agree on paper, so they decided to have a fistfight in the parking lot next to TikTok (but not owned by TikTok). So sometimes there needs to be a third observing neutral party to weigh all the options. This is where the challenger framework comes in.

This challenger pattern isn't new. The academic literature calls it multi-agent debate, or the 'society of minds' approach (Du et al., 2023). The numbers are good. Adaptive heterogeneous debate frameworks report 4 to 6% higher accuracy and over 30% fewer factual errors compared to standard debate (Zhou and Chen, 2025). More recent work on triggering debate only when it helps shows up to 13.5% accuracy gains while cutting token cost by 92% (Fan et al., 2025). The intuition matches the data: a model that has to defend its answer against a skeptic produces better answers.

I get upset when people get angry at LLMs for hallucinating and decide to distrust them entirely. Humans hallucinate too. Right now, I could flip the table and walk out of the office. Anyone really could. The reason humans don't do this isn't physical. It's that we have societal rules, master guidelines, that govern our behavior.

LLMs need those master guidelines too. But here's where it gets hard. An LLM has much less working memory than a human. It can't hold every relevant societal rule in active context at once. It has the gist of things, the same way a doctor has the gist of most situations but reaches for the textbook when the details matter. If someone asked you to solve a calculus problem from high school, would you trust yourself to get it right without going back to the textbook? (Unless you're a calculus teacher, in which case, fine, lol.)

The problem is: a human will say 'I don't remember, let me look it up.' An LLM will say it knows, and then be confidently wrong. The calibration research is blunt about why. From a recent paper on LLM certainty robustness: 'These models are not trained to say "I don't know." In fact, admitting uncertainty is usually penalized during training.' The training objective rewards giving an answer. Saying 'I'm not sure' is a worse-scoring response in the data the model learned from. So the model defaults to confidence even when it shouldn't.

That overconfidence is well-documented. Stanford research on confidence calibration found that nominal 99% confidence intervals from frontier LLMs cover the true answer only 65% of the time (Epstein et al., 2025). That's not 'a few hallucinations.' That's systematic overconfidence baked into the way models are trained.

Which is why the challenger matters. LLMs are usually optimized to answer the question. That's priority one. The internal weighting on 'say you're unsure' isn't high enough to override that default. So you introduce the doubt from outside: a second LLM whose job is to disbelieve everything, demand citations, and verify the citations actually exist and say what they're claimed to say.

Here's a thought experiment.

Ask an LLM to only say the word 'alphabet.' It will say 'alphabet.' Ask a human to only say the word 'alphabet.' They'll say 'alphabet.' Both might occasionally produce something like 'I guess I can say the word alphabet,' which is a small hallucination relative to the instructions, but most of the time, both comply.

Now ask both to solve a complex calculus problem with no support. Say, what's the derivative of ln(2354x)/log(67x)? I don't even know the answer to that off the top of my head. The LLM will provide an answer immediately and confidently. A human without recent calculus experience will guess their way through it. Both might get the order of steps wrong. Both will probably be wrong.

Now give them both a textbook with the right steps, the right order, and a calculator. They both probably get it right.

But here's the difference. An LLM shouldn't be given that supporting information as more text inside its context window. That eats memory. It should have it as a database, something it can query and cite without keeping the whole textbook open.

The buzzword for this is 'RAG,' or retrieval-augmented generation. Strip the acronym and it's exactly what it sounds like: an LLM attached to a database. The original 2020 paper from Lewis et al. at Facebook AI called this combining 'parametric memory' (what's in the model's weights) with 'non-parametric memory' (what's in a queryable index). The papers are full of acronyms, but the idea is just: connect the model to a real source of truth and let it look things up.

The numbers back up the intuition. A hypergraph-driven RAG system delivered a 12.3% accuracy gain over direct LLM use on a medical dataset, and held its accuracy as queries got harder, where plain retrieval degraded (Feng et al., 2026). The math is consistent: the LLM gets dramatically more reliable when it doesn't have to remember everything itself.

Humans already have this. Our brains have an automatic database system. A few months ago, I was at an Italian cafe with my mom and took a bite of green ice cream. I got an instant flashback to my childhood in New Jersey. I used to eat this, but I'd never known what it was called. It was spumoni. The taste was the key, and my brain went straight to the database and pulled the memory.

The research community has a name for this too. The cognitive science literature calls it cognitive offloading, where humans rely on external props, tools, and sensory cues to extend the working memory in our skulls (Gilbert et al., 2023). Our brains aren't designed to hold everything. They're designed to know where to look.

LLMs need that same architecture to behave like a human brain. Not more text. Not bigger context windows. A real, queryable, cite-able memory store, with the right routing agent in front of it, and a challenger LLM keeping it honest.

So here's what I'd tell my family the next time it comes up. They're right that AI on its own makes things up. They're right that people who trust it blindly are going to get burned. But the version of AI in those headlines is the version that's been asked to act like a brain. That's not what it's for.

An LLM isn't supposed to know everything. It's a connector. It pulls the parts together and tells you the story. The intelligence is in the architecture. The LLM is just the voice.

That's what I'm building.