Agents don't fail like services fail. And we're not tooled for it.
I built a multi-agent stock recommender. One agent reads news. One runs technical and fundamental analysis. A coordinator queries both, reconciles their rankings, and explains its reasoning.
It works. Building it also made something uncomfortable obvious.
When a service dies, it dies loudly
When a microservice dies, it dies loudly. You get a 500. Alerting fires. The circuit breaker trips. Thirty years of distributed systems practice exists for exactly this.
When an agent fails, it often keeps going.
The news agent returns something plausible but wrong. The coordinator has no way to know. It synthesizes confidently on top of garbage — and the output looks exactly like a good output.
Or the coordinator sits waiting on a specialist quietly stuck in a reasoning loop. No exception. No error. Just tokens burning and nothing coming back.
Silent failure with a confident face is the hardest failure mode there is.
"But the tooling exists" — not quite
Portkey and LiteLLM make model calls reliable: routing, caching, fallbacks. LangSmith measures quality. Neither knows what a reasoning loop is, or what it costs you when one happens.
The gap is the agent loop itself. So I'm building for it.
A gateway — sits in the request path
Verifier hooks before output reaches the caller. Loop detection and circuit breaking. Hard cost caps per session. The point is to put something in the path that can say no before a confident-but-wrong answer, or a runaway loop, ever reaches the caller.
A harness — breaks agents on purpose
Tool timeouts, malformed responses, forced loops. It measures recovery rate, p95 under fault, and the number nobody publishes: what a failing agent actually costs you.
Envoy and AWS FIS, for agents.
Where I'm genuinely unsure — and where I'd want to be told I'm wrong
- Loop detection heuristics. State repetition is the obvious signal — but agents legitimately revisit states. Where's the line before you start killing healthy runs?
- Verifier placement. Per tool call is expensive. Only at the end is too late. Is there a principled middle?
- Cost attribution in multi-agent topologies. When a coordinator's spend explodes, which specialist actually caused it?
Building this in public
I'm building this over the next month, in public. If you've run agents in production and hit these — I want to hear where my design breaks.
