~6 min read
Nikhil Bhardwaj — AI Product Manager at DataDrew. Field notes on shipping AI agents, evals, and product design. LinkedIn · About

MIRA — the chatbot inside MyGate, the app I use to manage gate approvals at my apartment — finally offered to raise a support ticket.

I had been arguing with it for eight minutes.

I'd asked something simple — when a delivery arrives at the gate, can only the person who ordered approve? Right now all the members of the family receive an approval ping.

MIRA's first answer: yes, all five will get it.

I pushed back: "I have 5 people registered — why do you want to disturb all 5 when just one person preferred something?"

It suggested a workaround — "members who don't want to be disturbed can turn off notifications."

I asked the obvious follow-up: if they turn it off, who approves?

MIRA repeated itself.

"Members who don't want to be disturbed can turn off notifications. The preferred member keeps theirs on. This way, at least the right person gets the approval request, while others are not disturbed."

— MIRA, MyGate Resident App

The same workaround. With more confidence.

That conversation ran for eight minutes and produced three distinct failure modes — each of which says more about how the agent was designed than how smart the model behind it is.

Here are the three.

Three failure modes I keep seeing in support bots

01 The Contradiction Loop

The bot suggested a workaround. I caught that the workaround broke its own promise — turn off notifications, but somehow also approve them. Instead of registering that I had just rejected its idea, MIRA rephrased the same workaround with a closing flourish: "at least the right person gets the approval request."

The bot didn't track that I'd already pushed back. Each turn was a fresh start. That isn't multi-turn — it's single-turn dressed up to look like a conversation.

Having the memory to keep the rejection on top of customer preference is probably the most important product feature, and somehow it's the first thing missing.

02 Empathy as Camouflage

"That's a great point!"

"That's a very valid concern! You're absolutely right."

"I completely understand your concern." 😊

These show up in MIRA's replies after every pushback.

The pattern: bot acknowledges the frustration with empathy filler, then delivers exactly the same answer that caused the frustration. By the third "I completely understand" after three failed answers, what the user actually feels is — this thing doesn't.

Empathy from a bot is performance. The user can tell. And once they can tell, the rest of the conversation turns adversarial — they're now arguing with the bot, not asking it for help.

By the third "I understand" after three failed answers, what the user actually feels is — this thing doesn't.

03 Your Problem, Not Ours

"Unfortunately, the MyGate app currently does not support enabling or disabling notifications on a per-member basis."

MIRA framed the gap as a fact for me to work around — not as feedback for the product team to act on.

It never said "I'll log this for the product team", or "want me to flag this as a feature request?" The support ticket only got offered after I explicitly asked for one — three exchanges later.

This is a product decision, not a model limitation. Whoever shipped MIRA chose not to wire feedback capture into dead-end conversations. That's a missed loop. Every dead-end chat is the most concentrated signal you'll ever get about what the product is missing — and right now it's disappearing into a "completed conversation" log.

Smaller things — worth listing

A few more things from the same eight minutes. None big enough to be a pattern, all big enough to add up.

What this says about agent design

The bigger lesson — agent creation isn't about handing a workflow to an agent and assuming it'll work. The workflow is the easy part. The hard part is everything around it.

Most chatbot launches I've seen optimise for the demo — a fluent, helpful-sounding answer to a clean question. Real users break that quickly. The eight-minute conversation with MIRA is the actual test, not the two-minute demo.

That's the gap. And it's not a model gap — it's a product design gap.

What I'd actually fix

If I were rebuilding MIRA, I'd start with one rule — the moment the bot says "the app does not support X", the rest of the conversation auto-converts into a structured feature request. Use case, frequency, severity, who asked. The user is told: "I've logged this. Here's what I sent."

That single change would do more for the product than any model upgrade.

None of the three failure modes need a smarter LLM to fix. They need a PM who sat with the bot for ten minutes and noticed.

The depressing part — these aren't edge cases. They're the modal experience. Most users don't fight their bots for eight minutes; they give up at minute two and try Google instead. The bot logged a "completed conversation" and the company filed it under "AI is working".

So the question I keep coming back to is —

How long do you usually argue with a chatbot before you give up and Google it?

Want to talk about this?

Saturday sessions on AI · open to corporates and colleges

Most Saturdays I block off time to talk about AI — how to think about agent design, what's actually worth shipping, where evals fit in, and where the gap between hype and product reality usually shows up.

I have been sharing sessions with larger teams implementing AI solutions, and also seminars with college students.

ai agents product management chatbots customer support field note analytics growth
More field notes: all writing · Caveman plugin saves tokens at an 18.6% rate · or see my work.