A brilliant demonstration tells you that an AI tool can succeed once. A small test set helps you learn where it succeeds, where it fails and how much checking your real work still needs.
The useful habit: test the tasks you actually do, with success criteria written before you inspect the answers.
Choose one job
Start narrowly: summarising meeting notes, extracting invoice fields or drafting replies from an approved policy. “Is this AI good?” is too broad. “Can it extract these five fields without inventing missing values?” is testable.
Imagine a fictional team that wants an AI tool to turn customer messages into short internal summaries. The important result is an accurate summary of the request, not a friendly response to the customer.
Collect a varied starter set
For a first manual exercise, you might choose ten to twenty carefully selected examples. That is a manageable starting point, not a statistically sufficient sample or a certification of reliability. Expand the set as you learn about the task.
| Case type | Example for message summaries |
|---|---|
| Ordinary | A clear request with one issue |
| Complex | Several issues and a long history |
| Missing information | A request without an order reference |
| Ambiguous | A pronoun that could refer to two products |
| Boundary | A message outside the team’s supported service |
| Misleading content | A quoted instruction that should be summarised, not obeyed |
Use fictional or properly authorised, sanitised material. Our document privacy checklist explains what to inspect before uploading.
Write the expected behaviour
You do not always need one perfect reference sentence. For a summary, define required facts, forbidden inventions and a length limit. For field extraction, record the correct values and specify how missing fields should appear.
A useful criterion is “mentions the delayed item and requested refund; does not claim the refund was approved.” A vague criterion such as “sounds professional” will not catch the most consequential error.
Separate critical failures from style issues
Use a simple rubric: pass, needs editing or fails. Add a separate critical-failure flag for problems such as fabricated commitments, exposed private information or following instructions embedded in untrusted source material.
Do not let a strong average hide an unacceptable failure. A tool that writes nine attractive summaries and invents an approval in the tenth may still be unsuitable for unattended use.
Keep the comparison fair
- Save the input, instructions and available reference material.
- Record the tool or model version where available and the test date.
- Run competing versions on the same cases.
- Score against the rubric before choosing a favourite.
- Record the actual mistake and its likely consequence.
- Repeat selected cases to look for variation in generated responses.
If you change the prompt, change one meaningful element at a time where practical. Otherwise you may know that performance changed without knowing why.
Keep some examples aside
Repeatedly rewriting a prompt to fit the same cases can produce an impressive score on those cases while leaving new work unreliable. Hold back some examples for a later check and add fresh real-world failure patterns, with appropriate privacy protection.
A starter set is especially limited when inputs differ by language, format, length or user group. A result on short English messages does not establish performance on long multilingual attachments.
Use the results to choose the workflow
The outcome may be “use for first drafts with human checking,” “restrict to simple cases” or “do not use for this task yet.” These are useful decisions. Testing is not only about selecting a winner.
Continue monitoring after deployment. New inputs, changed tools and revised instructions can create failures that the original examples did not reveal. Review the set when the task changes, rather than treating a past score as permanent.
Note to keep: a small AI test set makes uncertainty visible. It reduces guesswork but does not eliminate the need for judgement.
Related: compare two AI answers and verify an individual response.
Sources
- Google Cloud: evaluating and operating generative-AI applications
- NIST AI RMF Core: measurement and ongoing evaluation
The starter sample size and scoring rubric here are practical examples, not thresholds prescribed by these sources.
