Why a general reviewer misses AI’s mistakes, what work to give the machine, and how to tell if your checks are real.
Putting a person between the AI and the customer sounds safe. It only works if that person can actually spot when the AI is wrong.
We asked Alex Shabala, Group Head of Data Science at Capitec Bank, who checks the AI at a bank serving around 26 million clients…
The move: an expert, not just a human
Capitec’s AI work is about giving staff more capacity, not replacing them, which Alex says has never been the bank’s culture. That makes the person in the loop the most important part of the system, and the bank is specific about who it should be.
“We have a policy of expert in the loop, not human in the loop. There has to be someone who knows about that specific domain.”
How to put an expert in the loop on your AI
1. Name the expert for each AI task
For every job you hand to AI, decide who checks it, and make it someone who knows that work. A fraud analyst checks fraud decisions; a credit specialist checks credit. A general reviewer can confirm the output looks tidy. Only an expert can tell whether it’s right.
2. Know why any human isn’t enough
People tend to trust confident, well-written AI output more than they should. It’s called automation bias, and it’s well documented: the reviewer approves without really checking, especially once the AI has been right many times. Europe’s AI Act now names the risk directly in its rules on human oversight. A reviewer without the specialist knowledge has little chance of catching the errors that matter.
3. Give the machine the routine work
Alex’s split is clear. Machines handle the routine, high-volume, monotonous work. People deal with ambiguity and the rare cases that don’t fit a pattern. Design your process so the expert spends their time on the cases that need judgment, not on re-reading everything the AI produces.
4. Check that the expert is still really checking
Watch how often your expert changes or rejects what the AI suggests. If that number falls close to zero in work where some disagreement is normal, the review may have become a rubber stamp. One simple test is to slip in a few cases where you know the AI’s answer is wrong, and see whether they get caught.
5. Aim for productivity, not fewer people
Alex says most people need their productivity multiplied by ten, and today’s tools get closer to two and a half to four times. Those are big gains, but his view is that AI will displace tasks, not jobs. The expert stays and gets far more done.
The big payoff
With the right expert in the loop, you get the speed of AI without handing it decisions nobody is equipped to check. Your specialists stop doing the monotonous work and spend their time where their judgment counts.
You might also be interested in
Want the full story?
Alex’s full panel from The Open Letter Cape Town 2026 is available to members inside the Founder Collab, where he goes further than we could cover here:
Why fraud tactics have a half-life of four to six weeks, and how Capitec keeps its models up to date
How Capitec writes scam warnings customers actually listen to
When AI hallucination is fine, and when it isn’t
Why he hires for curiosity over credentials
You’ll also get access to 40+ masterclasses from SA founders and operators on sales, fundraising, UX, paid media and more inside The Founder Collab.
Get more SA tech and business news, tips and business-building workflows, and subscribe to The Open Letter.
