How to Build a Small Test Set for an AI Tool

A brilliant demonstration tells you that an AI tool can succeed once. A small test set helps you learn where it succeeds, where it fails and how much checking your real work still needs.

The useful habit: test the tasks you actually do, with success criteria written before you inspect the answers.

Choose one job

Start narrowly: summarising meeting notes, extracting invoice fields or drafting replies from an approved policy. “Is this AI good?” is too broad. “Can it extract these five fields without inventing missing values?” is testable.

Imagine a fictional team that wants an AI tool to turn customer messages into short internal summaries. The important result is an accurate summary of the request, not a friendly response to the customer.

Collect a varied starter set

For a first manual exercise, you might choose ten to twenty carefully selected examples. That is a manageable starting point, not a statistically sufficient sample or a certification of reliability. Expand the set as you learn about the task.

Case type Example for message summaries
Ordinary A clear request with one issue
Complex Several issues and a long history
Missing information A request without an order reference
Ambiguous A pronoun that could refer to two products
Boundary A message outside the team’s supported service
Misleading content A quoted instruction that should be summarised, not obeyed

Use fictional or properly authorised, sanitised material. Our document privacy checklist explains what to inspect before uploading.

Write the expected behaviour

You do not always need one perfect reference sentence. For a summary, define required facts, forbidden inventions and a length limit. For field extraction, record the correct values and specify how missing fields should appear.

A useful criterion is “mentions the delayed item and requested refund; does not claim the refund was approved.” A vague criterion such as “sounds professional” will not catch the most consequential error.

Separate critical failures from style issues

Use a simple rubric: pass, needs editing or fails. Add a separate critical-failure flag for problems such as fabricated commitments, exposed private information or following instructions embedded in untrusted source material.

Do not let a strong average hide an unacceptable failure. A tool that writes nine attractive summaries and invents an approval in the tenth may still be unsuitable for unattended use.

Keep the comparison fair

  1. Save the input, instructions and available reference material.
  2. Record the tool or model version where available and the test date.
  3. Run competing versions on the same cases.
  4. Score against the rubric before choosing a favourite.
  5. Record the actual mistake and its likely consequence.
  6. Repeat selected cases to look for variation in generated responses.

If you change the prompt, change one meaningful element at a time where practical. Otherwise you may know that performance changed without knowing why.

Keep some examples aside

Repeatedly rewriting a prompt to fit the same cases can produce an impressive score on those cases while leaving new work unreliable. Hold back some examples for a later check and add fresh real-world failure patterns, with appropriate privacy protection.

A starter set is especially limited when inputs differ by language, format, length or user group. A result on short English messages does not establish performance on long multilingual attachments.

Use the results to choose the workflow

The outcome may be “use for first drafts with human checking,” “restrict to simple cases” or “do not use for this task yet.” These are useful decisions. Testing is not only about selecting a winner.

Continue monitoring after deployment. New inputs, changed tools and revised instructions can create failures that the original examples did not reveal. Review the set when the task changes, rather than treating a past score as permanent.

Note to keep: a small AI test set makes uncertainty visible. It reduces guesswork but does not eliminate the need for judgement.

Related: compare two AI answers and verify an individual response.

Sources

The starter sample size and scoring rubric here are practical examples, not thresholds prescribed by these sources.

Latest

How to Make a Decision When Important Information Is Missing

Waiting for complete information can become a decision of...

Why Every Process Needs a Way to Catch Mistakes

A careful person can still attach the wrong file,...

How to Run a Small Experiment Before Making a Big Change

A new system can look convincing in a meeting...

How to Write Instructions Someone Else Can Actually Follow

“Put the files in the right folder and send...

Newsletter

spot_img

Don't miss

How to Make a Decision When Important Information Is Missing

Waiting for complete information can become a decision of...

Why Every Process Needs a Way to Catch Mistakes

A careful person can still attach the wrong file,...

How to Run a Small Experiment Before Making a Big Change

A new system can look convincing in a meeting...

How to Write Instructions Someone Else Can Actually Follow

“Put the files in the right folder and send...

Why a Lower Monthly Payment Can Still Cost More Overall

A monthly payment answers one important question: what must...

How to Make a Decision When Important Information Is Missing

Waiting for complete information can become a decision of its own. Deadlines pass, options disappear and the cost of delay grows. Acting immediately can...

Why Every Process Needs a Way to Catch Mistakes

A careful person can still attach the wrong file, miss a field or send a package to the wrong destination. A dependable process assumes...

How to Run a Small Experiment Before Making a Big Change

A new system can look convincing in a meeting and become awkward in daily use. A small experiment gives you a way to learn...