The thing nobody tells you about building agents is that half the job isn't even the reasoning. It's plumbing. It's the SDK that changed its API between two minor versions. It's the tool call that returned a shape you didn't expect. It's the hackathon at 2am where you're staring at an AttributeError that didn't exist yesterday.
I started getting into agents properly at IBM's 48 hour WatsonX hackathon. Oba and I submitted ArcTan, a code review tool that takes a zipped Python project, walks the file tree and pushes it through a multi agent system inside WatsonX Orchestrate. One agent for static analysis, one for dependency smells, one for writing the actual review notes a human would care about. It was the first time I actually got the difference between "LLM that calls functions" and a real orchestrated system where agents pass structured state between each other. Came out of that weekend wanting to build more of it.
We lost. It was a virtual submission so there was no stage, no trophy moment, just an email that took way too long to arrive. 1000+ participants and we didn't place top 3. First place went to A Team for NYC Property Scout, an agent that pulls thousands of public records into a Transparency Report Card for renters and buyers. Actually a solid idea. Second was simply, a decision tracking agent for Slack so customer impacting calls don't get buried. Third was Suyogs1, some governance driven incident response thing. I watched all their demos and felt that specific type of bad where you can see exactly why you lost and also refuse to admit it at the same time.
So when Babcock ran their Tech Week hackathon I went back in with something to prove. Eave Health wasn't an idea I'd been sitting on either. We brainstormed it the day before submission. Literal whiteboard session the evening before the deadline, landed on the pitch, started coding. The idea was a health intelligence platform sitting between patients and hospitals. Unified medical records as a single source of truth for diagnosis, agentic post visit care, proactive check ins, next of kin escalation when a patient goes quiet and personalised diabetes and hypertension risk scoring on top of all of that.
Basically I wanted to see if the orchestration stuff from ArcTan would hold up in a domain where being wrong actually hurts someone. Healthcare track, trust problem, the whole thing. I went in thinking I'd split Eave into a multi agent system too because that's what sounds good when you say it out loud. I left thinking most people reaching for multi agent don't actually need it. We won that one, which helped the part of my brain still annoyed about IBM.
Reading the room before touching code
Between the two hackathons I went deep on other people's thinking. IBM's Technology Untangled and a bunch of the IBM Techsplainer episodes on agentic AI were on repeat anytime I was walking or commuting. Some of it was marketing, sure, but the parts where they broke down how an agent actually plans, when it should defer to a tool and where the handoff between reasoning and execution sits, that stuff stuck.
Then I read through Anthropic's Building effective agents piece. Honestly one of the cleanest intros I've seen to the whole space. They draw a hard line between workflows, which are basically pipelines you've pre decided and agents, which get to choose their own path. The point they kept hammering is that you should start simple and only add agent like behaviour when the problem actually needs it. Most of the time a workflow with a couple of tool calls is all you need and reaching for full autonomy just makes things harder to debug.
That framing was the thing that broke my multi agent instinct. I'd been treating "more agents" as a signal of technical depth. Anthropic basically said the opposite. The depth is knowing when not to reach for it. I kept that in my head the whole time I was building Eave.
Single agent, many tools. Stop overcomplicating
ArcTan genuinely needed multiple agents. Static analysis has different failure modes from dependency review and the review aggregator had to reconcile both before emitting anything. WatsonX Orchestrate makes that pattern cheap, you get proper handoffs, typed state and the agents can actually disagree with each other in useful ways. When I started Eave I assumed I'd port that architecture over. Onboarding agent, analysis agent, scheduler, outreach agent. Felt clean on paper.
Then I sketched the call graph and realised there was no point where two of them actually needed to reason concurrently. It was all sequential. Institution payload lands, something classifies it, a tool gets called, a response goes out. Patient messages the agent, intent gets extracted, appointment gets booked. No race conditions, no disagreement to reconcile, no parallel streams of thought. That's not multi-agent, that's one orchestrator with a decent tool belt and enough state to know where it is in the pipeline.
The real win was collapsing everything into a single LLM loop with functions like ANALYZE_PRESCRIPTION, SCHEDULE_APPOINTMENT, COMPOSE_CHECK_IN, ESCALATE_TO_NOK. The agent doesn't need a personality split. It just needs to know which tool to reach for and when, with the patient's state as context. I wasted the first few hours of the hackathon on architecture that would have looked great in a pitch deck diagram but would have been miserable to debug live on stage. The heuristic I've settled on: if you can't name a moment where two agents need to think at the same time about different things, you don't need two agents.
The tool layer is where everything breaks
Reasoning is the fun part. Tool calls are where you lose your weekend. I was using Composio for Gmail integration and the SDK had one API shape in one project and a completely different one in another. Same package name, different versions, different everything. composio.client.tools.execute here, composio.actions.execute there and nothing in between worked.
Lesson I actually learned: pin your versions. Every tool integration in an agent pipeline is a fragile edge. If you don't lock the package version, you're one pip install away from a broken demo. For Eave I eventually just copied the exact versions from my other working project and stopped guessing.
State is the thing multi-agent people are actually missing
When people say they need multi-agent, nine times out of ten what they actually need is better state management. Who's in the middle of onboarding. Who has an active prescription. Who hasn't replied in 72 hours. That's not more agents, that's a store with some structure and a pipeline that checks it.
Eave's orchestrator holds patient state in memory for the demo and each entry point, a new message, an institution payload, a scheduled tick, just reads state and decides what to do next. No agent-to-agent chatter. No coordination overhead. The "intelligence" is in the routing, not in the number of LLM calls.
Temperature matters more than you'd think
I had the model cranked up to default temperature on the first build and the check-in messages came out sounding like a chatbot from 2021. Exclamation marks, emojis, the whole deal. Dropped it to 0.3, added tighter system prompts with explicit bad examples and the tone calmed down immediately. In healthcare especially, you don't want "Hey!! How's your week going?!" You want something that reads like a person who's paying attention but isn't trying too hard.
What I'm still chewing on
Agents are good at picking the right tool. They're bad at knowing when to stop. I keep running into loops where the model wants to "verify" something one more time, or "just check" another source and it burns through tokens on work the user never asked for. Hard-capping the tool call count helps. So does making the system prompt explicit about when to return without acting.
The other thing is failure recovery. When a tool call fails, the agent's instinct is to retry or hallucinate around it. Neither is what you want. What you want is for it to stop, surface the error and let you decide. That's a whole design problem I haven't cracked yet. For now I just wrap every tool call in try/except and return the exception string into the model's context so it can at least tell the user something went wrong instead of pretending everything's fine.