The outcome, the five users and the engagement signal that tells you whether to build or kill the feature. Building AI features has never been faster, but the question that gets skipped: would anyone use the outcome this AI feature produces? 

To know if you need an AI feature, we got Hilde Franzsen, Brand and Marketing Director at Stellenbosch UX and brand agency Inkblot, to show us how to get sure

It’s called the Wizard of Oz test. The user thinks the AI is doing the work, but actually, it’s just you (a human) doing it by hand in the background.

“Before you write a single line of code, do what the AI would do manually for five users. Send the recommendation yourself. Write the summary yourself. If a user doesn’t engage when a human delivers it, they’re not necessarily going to engage better when a machine delivers it.”

“AI chatbot” is a feature. “An answer to their tax question in under a minute” is an outcome. Users care about the outcomes, so focus on that: If you can’t describe the outcome in one sentence, the feature isn’t ready to test, let alone build.

Not friends or your team, but real target users with the real problem. Five is enough. You’re looking for a binary signal (do they want the outcome or don’t they), not a statistically significant sample.

This is the Wizard of Oz move. Pretend you’re the AI: Answer their questions yourself, write the summary, send the recommendation – whatever they ask for.

The user doesn’t need to know its manual; all you’re testing is whether they engage with and want the outcome.

Engagement is when they come back asking for more, act on it or tell someone else. That’s real demand for the outcome. Saying “thanks, that’s interesting” means you can discard it.

If four out of five users really engage with the human version, you can build the AI feature. If they don’t, you just saved yourself many hours of work that no one wanted anyway.

Building the AI feature is the expensive part. The Wizard of Oz test takes an afternoon. The cheapest test is always the one you do before you build.

This test is one piece of From Solutioneering to UX: How to Prioritise Designing the Right Things, Hilde’s full masterclass inside the Founder Collab, going deep on how to stop building the wrong things, including:

You’ll also get access to 40+ other masterclasses from SA founders and operators on sales, fundraising, paid media, automations and more inside The Founder Collab.

Get more SA tech and business news, tips and business-building workflows and subscribe to The Open Letter.