Before You Put an AI Chatbot in Front of Customers: A Launch Checklist for Mid-Market Owners and Operators

Before You Put an AI Chatbot in Front of Customers: A Launch Checklist for Mid-Market Owners and Operators

2026-10-27

Takeaway: A customer-facing AI chatbot speaks for your company, and a tribunal has already held a business to what its bot said. Before launch, decide what the bot may answer, ground it in your real policies, test it like a hostile customer would, give people an easy way to reach a human, and assign someone to read its conversations every week.

AI chatbots are one of the most tempting AI projects for a mid-market business. Support costs drop, customers get answers at 2 a.m., and the vendor demo looks great. But a chatbot is different from most internal AI tools in one important way: when it’s wrong, a customer sees it first, and the customer may act on it.

Three lessons from bots that went wrong

Your bot’s answers are your answers. In Moffatt v. Air Canada, 2024 BCCRT 149 (February 14, 2024), British Columbia’s Civil Resolution Tribunal looked at a chatbot that told a grieving customer they could apply for a reduced bereavement fare retroactively, within 90 days of buying the ticket. Air Canada’s own bereavement page said otherwise. Air Canada argued it couldn’t be held liable for information from the chatbot. Tribunal member Christopher C. Rivers called that “a remarkable submission” and wrote that “it should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.” He ordered Air Canada to pay CA$812.02. Small dollars, but a clear principle, and the case is from one Canadian small-claims tribunal, not a court of appeal.

Confident and wrong is the default failure. In an investigation published March 29, 2024, The Markup found that New York City’s MyCity chatbot, built to help business owners, told testers that employers could take a cut of workers’ tips and that restaurants could go cash-free. Both answers were wrong under city law. The bot sounded authoritative the whole time.

Updates can break a working bot. Delivery firm DPD said it had run an AI element in its chat successfully for years, until “an error occurred after a system update.” A frustrated customer then got the bot to swear and write a poem about how bad the company was. DPD disabled the AI element, the BBC reported on January 19, 2024.

The launch checklist

1. Pick a narrow job

Start with questions where the right answer is written down and rarely changes: order status, store hours, return windows, how to reset a password. Keep the bot away from refunds, pricing exceptions, legal, medical, or anything where a wrong answer creates an obligation, until you’ve seen it perform.

2. Ground it in your real policies, and keep them current

Most business chatbots answer from documents you feed them. If your policy pages disagree with each other, the bot will too. Air Canada’s bot and its bereavement page said different things. Pick one source of truth for each policy and name an owner who updates it.

3. Test it like a hostile customer

The OWASP Gen AI Security Project, a nonprofit security community, lists prompt injection as the first entry (LLM01) in its 2025 Top 10 for large language model applications. Prompt injection means a user’s input changes how the bot behaves in ways you didn’t intend. In the BBC’s account, the DPD customer simply asked the bot to “exaggerate and be over the top in your hatred.” OWASP says it’s unclear whether there are fool-proof ways to prevent it. Its recommended mitigations include constraining what the bot is allowed to do, filtering inputs and outputs, giving the bot only the access it needs, requiring human approval for high-risk actions, and running adversarial testing before and after launch. So before go-live, have staff try to make the bot promise discounts, badmouth you, or reveal information it shouldn’t.

4. Give the bot as little access as possible

A bot that can only read public help articles can embarrass you. A bot connected to your order system, customer records, or email can do real damage. Add each connection deliberately, and keep actions like refunds or account changes behind a human.

5. Make the human exit easy

DPD’s customer started experimenting with the bot after finding it, in his words to the BBC, “utterly useless at answering any queries.” Put “talk to a person” on screen, and make the bot hand off when it isn’t sure.

6. Re-test after every change

DPD’s problem started with a system update. Treat new models, new prompts, and new documents like a software release: run your test questions again before they go live.

7. Read the transcripts

Assign an owner to review a sample of conversations every week, track wrong answers, and fix the source document, not just the bot. That’s how you catch the MyCity-style errors before a reporter does.

The bottom line

A chatbot can earn its keep in customer service, but only if you treat it like a new employee who talks to customers: a defined job, approved material, supervision, and someone accountable for its mistakes. Yellow Coop’s CTO work includes an AI adoption plan tied to hours saved and headcount, not demos. Its CISO work includes an AI use policy that spells out which tools, which data, and who approves. Let’s talk.

Sources