Research & Insights3 min read

An Average AI Test Score Can Hide the Cases That Matter

A strong aggregate result can conceal a weak, high-consequence slice. Test AI by task, context and impact before the average becomes the decision.

Bokili Editorial· Verified November 7, 2026
ShareX
One strong average AI evaluation score splits into four passing task slices and one high-consequence failing slice

A single AI test score is easy to repeat: 91% accurate, 4.3 out of 5, or “passed 46 of 50 cases”. It is also easy to misuse. The average may combine routine and consequential tasks, common and rare inputs, clear and ambiguous requests, or people whose needs are not equally represented. Four strong slices can make one weak slice disappear.

The answer is not to reject averages. It is to make them earn their place. Before a score informs a rollout, break the evaluation into the cases where performance, consequence or user experience may differ. NIST’s AI Risk Management Framework calls for realistic test sets, documented conditions and limits on generalisability. UK evaluation guidance likewise warns that AI impacts can vary by task, context and group. A result without those conditions is a headline, not yet a decision.

The average answers the wrong question

Aggregate-only testSlice-aware test
QuestionHow did the system perform overall?Where does performance change enough to alter use?
Test setOne blended sampleNamed task, context and consequence slices
ResultOne mean or pass rateOverall result plus each material slice
DecisionDeploy or rejectDeploy, limit, redesign, add review or stop

Suppose an internal assistant answers 100 policy questions correctly 88 times. That sounds useful. But the 100 cases contain 80 common questions, 15 regional variations and five urgent exceptions. If it answers 76 common questions, ten regional questions and only two urgent exceptions correctly, the overall result remains 88%. The slice that could cause the greatest harm is the weakest one. More of the easy cases would make the average look better without improving that weakness.

Use the SLICE review before you trust the score

SLICE: five questions for a consequential evaluation

1

S — State the decision

Write what the result will authorise: pilot, limited use, broader deployment, mandatory review or no use.

2

L — List material slices

Name task types, languages, input conditions, user contexts and consequence levels where performance could differ.

3

I — Inspect each slice

Report the sample, metric, uncertainty and failure pattern for every material slice, not only the combined score.

4

C — Compare consequence

Do not weight a rare, high-impact case as though it were interchangeable with a routine one.

5

E — Establish the boundary

Set the route for weak or untested slices: human review, narrower scope, more testing, redesign or stop.

Reading is a start. Practice makes it stick.

Start learning

SLICE is a decision aid, not a statistical shortcut. Some slices will be too small for a stable percentage. Say so. Use qualitative review, targeted stress tests and domain experts rather than inventing precision. The Model Cards research proposed reporting performance across relevant conditions and groups precisely because a model’s suitability depends on intended use. The UK AI Playbook also treats testing, validation and ongoing monitoring as lifecycle work, not a one-off launch gate.

Worked example: policy answers with uneven consequences

A company tests an AI assistant on expense-policy questions. The common slice covers meal limits, receipts and mileage. The regional slice covers different currencies and local tax rules. The exception slice covers medical travel, accessibility needs and emergency bookings. The assistant does well on common questions, sometimes merges regional rules, and often omits the approver for exceptions. The owner does not “average away” the problem. They allow the tool to draft answers for common questions, require a source citation and human check for regional questions, and route exception questions directly to the responsible team until a new test demonstrates reliable coverage.

What to record beside any AI test score

  • The intended decision and named owner
  • The conditions represented in the test set
  • The material slices and why they matter
  • Sample size and uncertainty for every slice
  • Failure types, not only pass percentages
  • Who or what is missing from the test
  • The operating boundary for weak or untested slices
  • The date, version and trigger for retesting

Run a ten-minute hidden-slice test

Break one result into the cases that matter
  1. Choose one AI evaluation result your team already uses.
  2. Write the work decision that the result is meant to support.
  3. Split the cases by task type, consequence and one context where performance may change.
  4. Recalculate or qualitatively review each slice separately.
  5. Circle the weakest or least-tested consequential slice.
  6. Write one boundary or additional test before the result can authorise broader use.

A responsible evaluation does not ask only whether the AI works. It asks where it works, for whom, under what conditions and what happens when it does not. Keep the aggregate score for orientation. Put the slices beside it for the decision.

Practise this in Bokili

Turn what you just read into a ten-minute mission.

  • Check the Numbers Before You Use Them

    Practise a second check for calculations and reported figures.

    Start learning
  • Separate Verification From the Decision

    Test whether a checked output actually changes reliance.

    Start learning
  • Run a Delayed Transfer Check

    See whether a skill survives a fresh case after training.

    Start learning

Sources

  1. AI Risk Management Framework Core — NIST AI Resource Center
  2. AI Risks and Trustworthiness — NIST AI Resource Center
  3. Guidance on the Impact Evaluation of AI Interventions — HM Treasury and Evaluation Task Force
  4. Artificial Intelligence Playbook for the UK Government — Government Digital Service
  5. Model Cards for Model Reporting — Google Research
ShareX

Reading is a start. Practice makes it stick.

Bokili turns skills like this into ten-minute missions for your whole team, with instant feedback and progress you can see.

Start learning

Keep reading