In 2024, Air Canada’s chatbot invented a refund policy, and a Canadian tribunal ordered the airline to pay when a customer relied on it. The company owns what its AI tells customers.
We asked Alex Shabala, Group Head of Data Science at Capitec Bank, how to keep made-up answers away from customers…
The move: accept the risk, then contain it
Alex doesn’t believe hallucination will go away with current methods. The risk is never zero. The useful question is what happens when your system does get something wrong, and how many safeguards sit between that mistake and a customer.
“At enterprise scale, it’s not a question of whether your system will fail. It’s when and how, and how you build defence in depth.”
How to stop AI hallucinations from reaching your customers
1. Accept that making things up is how it works
A language model squeezes a huge amount of human knowledge into something small enough to run on a laptop, and Alex points out that this compression loses detail. The same quality that makes it creative also makes it invent things. In his words, creativity and hallucination are 2 sides of the same coin. It’s not good or bad; it just is.
2. Use a different tool when you need reliability
If an answer must be right every time, such as a balance, a price or a policy, Alex’s advice is to reach for a different tool: rules or classic machine learning, which give predictable results. Use generative AI where some creativity is welcome. We covered how Capitec chooses between them.
3. Ground answers in your own information
For the jobs in between, connect the AI to your own documents so it answers from them instead of from memory. This approach, called retrieval-augmented generation or RAG, lowers the risk. Alex is clear that it doesn’t remove it, so it’s one layer, not the whole answer.
4. Build more than one layer of defence
Defence in depth means several checks, so one failure doesn’t reach the customer. Limit what the AI is allowed to answer, check any fact that matters, such as prices and policies, against your real records before it’s shown, and route high-stakes answers to an expert in the loop. Each layer catches some of what the last one missed.
5. Let your team see it get things wrong
Alex says the hype fades quickly once someone has seen AI make a mistake first-hand. So Capitec puts AI tools in as many hands as possible and lets people learn its limits. His description: it’s not magic, but it’s not useless. It’s a useful idiot. A team that knows that is far better at spotting errors before customers do.
The big payoff
You get the speed and flexibility of AI without betting your reputation on it never being wrong. Mistakes still happen, but they get caught inside the business instead of in a customer’s inbox.
You might also be interested in
Want the full story?
Alex’s full panel from The Open Letter Cape Town 2026 is available to members inside the Founder Collab, where he goes further than we could cover here:
Why fraud tactics have a half-life of 4 to 6 weeks, and how Capitec keeps its models up to date
How Capitec writes scam warnings customers actually listen to
Why Capitec trains some of its own models on data it would never send outside the bank
Why he hires for curiosity over credentials
You’ll also get access to 40+ masterclasses from SA founders and operators on sales, fundraising, UX, paid media and more inside The Founder Collab.
Get more SA tech and business news, tips and business-building workflows and subscribe to The Open Letter.



