An Average AI Test Score Can Hide the Cases That Matter
A strong aggregate result can conceal a weak, high-consequence slice. Test AI by task, context and impact before the average becomes the decision.

A single AI test score is easy to repeat: 91% accurate, 4.3 out of 5, or “passed 46 of 50 cases”. It is also easy to misuse. The average may combine routine and consequential tasks, common and rare inputs, clear and ambiguous requests, or people whose needs are not equally represented. Four strong slices can make one weak slice disappear.
The answer is not to reject averages. It is to make them earn their place. Before a score informs a rollout, break the evaluation into the cases where performance, consequence or user experience may differ. NIST’s AI Risk Management Framework calls for realistic test sets, documented conditions and limits on generalisability. UK evaluation guidance likewise warns that AI impacts can vary by task, context and group. A result without those conditions is a headline, not yet a decision.
The average answers the wrong question
| Aggregate-only test | Slice-aware test | |
|---|---|---|
| Question | How did the system perform overall? | Where does performance change enough to alter use? |
| Test set | One blended sample | Named task, context and consequence slices |
| Result | One mean or pass rate | Overall result plus each material slice |
| Decision | Deploy or reject | Deploy, limit, redesign, add review or stop |
Suppose an internal assistant answers 100 policy questions correctly 88 times. That sounds useful. But the 100 cases contain 80 common questions, 15 regional variations and five urgent exceptions. If it answers 76 common questions, ten regional questions and only two urgent exceptions correctly, the overall result remains 88%. The slice that could cause the greatest harm is the weakest one. More of the easy cases would make the average look better without improving that weakness.
Use the SLICE review before you trust the score
SLICE: five questions for a consequential evaluation
S — State the decision
Write what the result will authorise: pilot, limited use, broader deployment, mandatory review or no use.
L — List material slices
Name task types, languages, input conditions, user contexts and consequence levels where performance could differ.
I — Inspect each slice
Report the sample, metric, uncertainty and failure pattern for every material slice, not only the combined score.
C — Compare consequence
Do not weight a rare, high-impact case as though it were interchangeable with a routine one.
E — Establish the boundary
Set the route for weak or untested slices: human review, narrower scope, more testing, redesign or stop.
Reading is a start. Practice makes it stick.
Start learningSLICE is a decision aid, not a statistical shortcut. Some slices will be too small for a stable percentage. Say so. Use qualitative review, targeted stress tests and domain experts rather than inventing precision. The Model Cards research proposed reporting performance across relevant conditions and groups precisely because a model’s suitability depends on intended use. The UK AI Playbook also treats testing, validation and ongoing monitoring as lifecycle work, not a one-off launch gate.
Worked example: policy answers with uneven consequences
A company tests an AI assistant on expense-policy questions. The common slice covers meal limits, receipts and mileage. The regional slice covers different currencies and local tax rules. The exception slice covers medical travel, accessibility needs and emergency bookings. The assistant does well on common questions, sometimes merges regional rules, and often omits the approver for exceptions. The owner does not “average away” the problem. They allow the tool to draft answers for common questions, require a source citation and human check for regional questions, and route exception questions directly to the responsible team until a new test demonstrates reliable coverage.
What to record beside any AI test score
- The intended decision and named owner
- The conditions represented in the test set
- The material slices and why they matter
- Sample size and uncertainty for every slice
- Failure types, not only pass percentages
- Who or what is missing from the test
- The operating boundary for weak or untested slices
- The date, version and trigger for retesting
Run a ten-minute hidden-slice test
- Choose one AI evaluation result your team already uses.
- Write the work decision that the result is meant to support.
- Split the cases by task type, consequence and one context where performance may change.
- Recalculate or qualitatively review each slice separately.
- Circle the weakest or least-tested consequential slice.
- Write one boundary or additional test before the result can authorise broader use.
A responsible evaluation does not ask only whether the AI works. It asks where it works, for whom, under what conditions and what happens when it does not. Keep the aggregate score for orientation. Put the slices beside it for the decision.
Practise this in Bokili
Turn what you just read into a ten-minute mission.
- Start learning
Check the Numbers Before You Use Them
Practise a second check for calculations and reported figures.
- Start learning
Separate Verification From the Decision
Test whether a checked output actually changes reliance.
- Start learning
Run a Delayed Transfer Check
See whether a skill survives a fresh case after training.
Sources
- AI Risk Management Framework Core — NIST AI Resource Center
- AI Risks and Trustworthiness — NIST AI Resource Center
- Guidance on the Impact Evaluation of AI Interventions — HM Treasury and Evaluation Task Force
- Artificial Intelligence Playbook for the UK Government — Government Digital Service
- Model Cards for Model Reporting — Google Research
Reading is a start. Practice makes it stick.
Bokili turns skills like this into ten-minute missions for your whole team, with instant feedback and progress you can see.
Start learningKeep reading

AI Training for HR: Keep Health Inference Out of Absence Summaries
Train HR teams to produce a minimal operational absence summary without exposing health details, inventing causes or letting AI make a people decision.

Allocate AI Licences to Task-Ready Work, Not Waiting Lists
Use four readiness gates to allocate scarce AI access to work with an approved input, clear outcome, review owner and support route.

AI Skills for Employees: Calibrate Confidence Before Checking
Train employees to record confidence before AI advice, verify the decisive claim and learn from the gap between prediction and evidence.