Perspectives4 min read

AI Often Reveals the Broken Workflow Before It Improves It

An AI pilot can expose missing sources, rules, owners and skills before it creates speed. Diagnose the gap before prescribing more training.

Bokili EditorialPublished October 15, 2026· Verified October 15, 2026

An AI diagnostic prism reveals tool, process, training and policy defects inside a tangled workflow.

The first weeks of an AI pilot can make work look worse. Exceptions multiply. Review takes longer than expected. Different employees get different results from the same tool. Leaders may conclude that the model is weak or that people need more prompt training. Sometimes they do. But an AI pilot often reveals a problem that was already present: the workflow depended on unwritten rules, improvised sources, invisible hand-offs or one expert who quietly repaired every case. AI has not created all that disorder. It has made the disorder observable.

Why an AI pilot can look worse before it gets better

NIST’s AI Use Taxonomy starts with human goals and outcomes, then describes a task as a combination of activities. That distinction matters. If a pilot measures only the model response, it can miss the surrounding work that determines whether the response becomes useful. The same answer may succeed with a complete brief, a named source and a clear reviewer, then fail when any one of those elements disappears.

The UK Government AI Playbook makes a similar practical point: define the goal, understand user needs and decide where AI should or should not be used. It also recommends meaningful human control at the right stages. Those decisions force a team to name things that a manual process may have left implicit. The resulting questions can feel like friction, but they are also evidence about how the work really operates.

Four signals the pilot may be revealing

Tool issue

The system cannot reliably perform the required activity under the tested conditions. Change the tool, narrow the task or strengthen the technical control.

Process issue

The input, source, hand-off, owner or acceptance rule is missing or inconsistent. Repair the workflow before asking people to prompt harder.

Training issue

The workflow is sound, but people cannot yet perform one observable behaviour such as framing, checking or escalating. Practise that behaviour.

Policy issue

The team does not know what is allowed, who has authority or which evidence standard applies. Clarify the rule and decision owner.

Worked example: the model did not create the missing rule

Imagine a finance team piloting AI to prepare short commentary on overdue invoices. The first results are inconsistent. One analyst uses the ledger export; another adds notes from email. Customer names do not match across files. Nobody has defined when a disputed invoice may be described as overdue, and the final approver changes by region. A quick diagnosis says the model is unreliable. A better diagnosis separates the signals: file matching is a process issue, the disputed-invoice rule is a policy issue, choosing permitted inputs is a training issue, and any remaining extraction failure may be a tool issue.

	Quick conclusion	Diagnostic reading
Different answers	The model is inconsistent.	Check whether the same brief, source and rule were used.
Slow review	AI saved no time.	Find the missing evidence or authority that makes review expand.
Many exceptions	Employees are using it badly.	Separate recurring process gaps from behaviours that need practice.
One expert fixes everything	Keep the expert in the loop.	Make the expert’s hidden rule visible and assign an owner.

Reading is a start. Practice makes it stick.

Start learning

Use the first twenty cases as workflow evidence

Do not treat every failed case as a prompt to rewrite. Keep a small diagnostic record for the first twenty representative cases. Record the intended outcome, the point where work slowed or returned, the missing condition and the next owner. The purpose is not to build a fault register against employees. It is to distinguish a system problem from a learning problem before money is spent on the wrong remedy.

A four-step diagnostic loop

1
Capture the observable moment
Write what happened without guessing the cause: a source was missing, a reviewer returned the draft or an exception had no owner.
2
Classify the gap
Mark tool, process, training or policy. Use two labels only when the evidence genuinely points to both.
3
Test the smallest fix
Change one condition: standardise the input, clarify one rule, practise one behaviour or narrow the AI activity.
4
Re-run a comparable case
Check whether the fix improves the same work outcome. If it does not, revisit the diagnosis rather than adding more instruction.

Do not make training carry every organisational defect

OPM’s training-needs guidance asks why a required behaviour does not exist and warns that training is not always the best—or only—solution. That is a useful discipline for AI adoption. A course cannot repair an inaccessible source, conflicting approval rules or an impossible service target. Training becomes valuable after the workflow names the behaviour a person can actually perform.

Run a ten-minute pilot diagnosis

Choose one recent AI-assisted case that was slow, returned or rejected.
Write the expected work outcome and the exact point where progress stopped.
Classify the primary gap as tool, process, training or policy.
Name one piece of evidence that supports the classification.
Choose the smallest fix and one comparable case for a re-test.
Assign an owner and a date for the re-test.

Visibility is an early return on the pilot

A pilot should still be judged against useful outcomes, not celebrated for producing problems. But newly visible problems are not the same as new problems. Leaders can use them to improve the workflow, focus training and set honest controls before scale. Continue with the AI workflow friction test (https://bokili.com/en/learn/ai-workflow-friction-test), the exception-queue business case (https://bokili.com/en/learn/ai-automation-exception-queue) and the workflow-constraint method (https://bokili.com/en/learn/ai-training-workflow-constraint). Bokili then gives teams short, role-relevant practice for the behaviours that remain. The first value of AI may be a better map of the work. Use that map before demanding speed.