When not to use AI.

The expensive AI mistakes we see are rarely technical. They are projects that should never have been AI projects — a rules problem, a data problem or a broken process, given a model instead of a fix. Here is the test we run before quoting anything.

1. Can you write the rule down?

If a competent new employee could follow a written rule to do this task correctly, you have a rules problem, and rules are cheaper, faster, auditable and deterministic. "Route the email to accounts if the subject contains an invoice number" is an if statement. It does not need a model, a token budget or an evaluation harness.

The genuine case for AI starts where the rule list stops converging — where every rule you add spawns two exceptions and nobody can enumerate them. That is a real signal. "We have a hundred rules and they mostly work" is not.

2. What happens when it is wrong?

This is the question that kills most candidates, and it should be asked first rather than last.

That last row is the dangerous one. A system whose errors are invisible does not get corrected; it gets trusted. Silent failure is the actual risk in automated work — considerably more than a wrong answer somebody spots.

3. Is 80% actually useful to you?

People hear "80% accurate" as "mostly done". For a workflow it often means zero value, because if you cannot tell which 80%, a human re-checks 100% — and you have added a step rather than removed one.

What rescues this is not higher accuracy, it is calibrated confidence: the system knowing when it does not know, and routing those cases to a person. A system that handles 60% of cases and correctly flags the other 40% is genuinely useful. One that handles 95% and cannot tell you which 5% it got wrong frequently is not.

4. Do you have the data, and is it any good?

Two separate failures hide behind one question.

• VolumeNot a problem for most modern applications — you are usually calling a general model, not training one. Be sceptical of anyone insisting you need a large proprietary dataset for a task a good prompt handles.
• TruthThe real problem. If your product data contradicts itself, your documentation is three years stale, or two systems disagree about the same customer, an AI layer will confidently repeat all of it — faster, and in a tone that sounds authoritative. Cleaning that up is the project. The AI is the easy part afterwards.

Wanting AI is a common way of discovering you have a data problem. That discovery is valuable, but it is a different project with a different budget.

5. Is the process actually broken?

Automating a bad process gives you a bad process that runs faster and is now harder to change. If your team spends four hours a week reformatting a report nobody reads, the fix is not an AI report generator.

Ask what happens to the output. If the honest answer is "it gets filed", stop there — you have found real savings and they cost nothing to realise.

6. What is the smallest version worth having?

If the only version that helps is the whole platform, the project is too big to start. Good AI work is narrow: one workflow, in production, measured against what it replaced.

A system that handles a single case reliably beats a platform that handles nine cases at 80%, for the reason in question three — nine unreliable capabilities means nine things a human still checks.

More on this