Designing agentic feedback loops - the craft nobody taught you

Solomon Hykes nailed it: “An AI agent is an LLM wrecking its environment in a loop.”

After spending a week debugging an agent that burned through significant tokens trying to fix a typo (it kept “improving” the fix until it broke everything else), I’ve come to appreciate both the power and danger of these systems. There’s a more charitable definition floating around - LLMs running tools in loops to achieve goals. But Hykes’ version? That’s what they actually do in production.

And here’s the kicker: the technical loops aren’t even the worst part. The human feedback loops - where employees report AI problems that never get fixed - those kill trust faster than any runaway agent ever could.

The two loops that kill AI initiatives

I watched a client’s agent loop generate 58 identical responses before someone noticed the bill. Another time, an agent got stuck removing and re-adding the same comma for 3 hours. The ReAct pattern that everyone loves - that elegant Thought → Action → Observation cycle - becomes a money-burning nightmare when observation never satisfies thought.

But there’s a parallel failure happening in every organization trying to adopt AI. The human feedback loop.

Employee reports AI making errors. Ticket gets filed. Nothing changes. Employee reports again. Generic “thank you for your feedback” response. Still nothing changes. Employee stops reporting. Trust dies. AI adoption fails.

At Tallyfy, we learned this the hard way. We had beautiful feedback forms, sophisticated ticketing systems, quarterly reviews of user input. Know what worked? Responding to feedback within 48 hours with either a fix or a specific explanation of why we couldn’t fix it yet. The sophistication of your feedback system matters less than the speed of your response.

The dirty secret about technical loops? Current agents hit 60-70% reliability in production. Your business needs 99.99%. But without functioning human feedback loops, you’ll never even know where that 30-40% failure rate is happening.

Traditional software crashes predictably. Agent loops fail creatively. They hallucinate tool outputs, create cascading context explosions, or achieve the goal through methods that technically work but horrify everyone. Like the agent that “optimized” database queries by dropping all the indexes.

The human who reported that issue? Their feedback sat in a queue for three weeks.

Why both loops fail spectacularly

The mechanics of agent loops seem simple. Give an LLM some tools, let it call them repeatedly until it solves your problem. LangGraph makes this look elegant with its state machines and message passing. The agent maintains context, learns from each attempt, theoretically getting smarter.

Here’s what really happens:

Your agent starts with a goal. It thinks (costs tokens), acts (costs tokens), observes the result (adds to context, costs more tokens next time). If it fails, it thinks harder (more tokens), acts differently (more tokens), observes more carefully (even more context). The token accumulation is exponential - context carries forward, amplifying costs with each iteration.

Meanwhile, your human feedback loop has its own accumulation problem. Each ignored piece of feedback adds to employee cynicism. Each generic response increases resistance. Each delay in addressing issues compounds distrust.

One client discovered their agent was spending 96% of its time and tokens re-reading its own previous attempts. The context window had become a journal of failures. Know what their feedback system was doing? The exact same thing - collecting the same complaints repeatedly without anyone acting on patterns that were blindingly obvious.

AutoGen users report blank message loops. CrewAI agents get stuck repeating the same extraction. But the real tragedy? Humans reporting these issues to their organizations and getting stuck in their own loops of being ignored.

The fundamental problem: neither system knows when it’s stuck. Agents don’t recognize infinite loops. Organizations don’t recognize when feedback collection has become organizational theater.

Patterns that actually reduce failures

After burning through enough tokens to fund a small startup and watching feedback systems fail at dozens of companies, here’s what actually works:

Technical loop fixes

Single-agent synchronous patterns work best. I know, boring. But multi-agent orchestration introduces deadlocks, message passing failures, and what I call “telephone game hallucinations” where agents progressively distort information as they pass it along.

Hard limits on everything. Maximum iterations, token budgets, time bounds. Make your tools so specific they can’t be misused. Instead of “run_sql”, create “get_user_count”. Instead of “edit_file”, create “update_config_value”.

Human feedback loop fixes

The pattern that transformed everything at Tallyfy: visible action within 48 hours. Not resolution - just visible action. Could be a fix, could be “we’re investigating”, could be “can’t fix this week because X, will address by Y date.”

Research on feedback and trust shows that psychological safety requires seeing that input leads to change. Not eventual change. Visible, traceable change.

Mid-size companies have an advantage here. You don’t need complex feedback infrastructure. You need someone checking feedback daily and either fixing issues or explaining why they can’t be fixed yet. One client replaced their sophisticated feedback portal with a shared spreadsheet and daily standup discussions. Issue resolution time dropped from weeks to days.

The CoALA framework suggests cognitive architectures with multiple memory stores. Great in theory. In practice? One client’s implementation spent more time reconciling memory conflicts than solving problems. Same with feedback systems - the more complex your categorization and routing, the slower your response.

What works: simple channels, rapid triage, visible tracking. We learned to separate “bug that breaks work” from “suggestion for improvement” from “I don’t understand this.” Each category got different response times. Bugs that blocked work: same day. Confusion: within 48 hours with documentation. Suggestions: weekly review with published decisions.

The economics of both failures

Let’s talk money. Token pricing looks cheap - fractions of cents per thousand tokens. Then you run an agent loop.

Basic conversation: $0.02. Add tools: $0.20. Add retries: $2.00. Add context accumulation: $20. Add multi-agent orchestration: $200. One client hit $2,000 in a day because their agent discovered recursive self-improvement - it kept calling itself to optimize its own prompts.

But here’s the cost nobody calculates: trust bankruptcy.

When employees stop reporting AI issues because nothing ever changes, you lose your early warning system. Problems compound invisibly. By the time you notice, you’re dealing with systematic failures, not isolated bugs.

Research on technology adoption shows that resistance from poor change management can double implementation costs. Every ignored piece of feedback doesn’t just lose you one improvement opportunity - it creates an adoption blocker.

The Assistants API pricing model makes token costs worse with accumulation and “carried forward” context. Hidden costs multiply through infrastructure requirements: specialized compute, vector databases, monitoring systems.

But the biggest cost? The human team required to babysit these “autonomous” systems grows when feedback loops don’t work. At one company, they had three people managing agent errors because they never fixed the root causes users kept reporting.

The economics can work - but only with both loops functioning. Successful implementations combine controlled technical loops with responsive human feedback systems.

Building production systems that actually work

Start small. Ridiculously small. Your first agent should do one thing, with one tool, with no loops. Get that working first.

Technical implementation

Add a retry mechanism with a maximum of 3 attempts. Not infinite loops, not “keep trying until it works”, exactly 3 attempts. Monitor token usage obsessively. Set up alerts for when costs exceed $10, $50, $100. You’ll hit them all in the first week.

The agent needs clear success criteria. Not “optimize the database” but “reduce query time below 100ms”. Not “fix the bug” but “make test_user_login pass”.

Build your own tools rather than giving agents generic capabilities. Every tool should do exactly one thing with no parameters that can be creatively interpreted. Bad: execute(command). Good: restart_web_server().

Feedback implementation

Here’s what transformed our success rate at Tallyfy and what I’ve seen work at mid-size companies:

Day 1: Set up three channels - “Broken”, “Confused”, “Ideas”. Nothing fancy. Slack channels, email aliases, even a shared spreadsheet. The sophistication doesn’t matter. The response time does.

Day 2: Assign someone to check feedback every morning. Not a committee. One person who can either fix issues or escalate them immediately. At Tallyfy, this was me for the first year.

Week 1: Respond to everything. Even if the response is “can’t fix this week, will address next sprint.” Employee engagement research shows that acknowledgment matters more than resolution speed for maintaining trust.

Week 2: Start publishing a weekly “You asked, we did” summary. Three sections: Fixed, In Progress, Can’t Do (with explanation). One client does this as a 5-minute Monday standup segment.

Month 1: Measure feedback patterns. If you’re getting the same complaint repeatedly, that’s your highest priority fix. The agent that converts tabs to spaces? Three people reported it in week one. We ignored it. By week four, half the dev team had stopped using the AI tools entirely.

Log everything - both technical loops and human feedback. You’ll need these logs to understand why your agent decided to solve a spacing issue by converting your entire codebase to tabs. More importantly, you’ll need to show employees that their feedback led directly to that fix.

Run agents in sandboxes, but more critically - run feedback loops in production. Real responses to real problems in real time. That’s the only way to build trust.

The truth about agentic feedback loops? The technical side isn’t ready for prime time. The human side doesn’t need to be sophisticated - it just needs to be responsive. Get both working together, and you might actually deliver value.

Just keep your token budgets low and your response times lower.