A model recognises a component perfectly in a bright product photograph. In the workshop, the component arrives at an angle, partly hidden by packaging and lit by a different lamp. Those are not minor details if that is where the system is supposed to work.
A useful test set asks whether the model can support the actual decision, not whether it can repeat an attractive demonstration. Begin by writing down the operating conditions and the mistakes that would matter most.
Define the decision before collecting pictures
Imagine an illustrative system that sorts metal brackets into two bins. What counts as a correct result: recognising the bracket type, spotting damage or locating the item for another machine? These are different tasks. A single headline accuracy figure does not explain which one was measured.
NIST’s discussion of AI risks and trustworthiness says accuracy measures should accompany realistic test sets representative of expected use and a description of the testing method. That principle is more useful than collecting images simply because they are easy to label.
Make a conditions list
Record the variations the system is expected to handle: lighting, camera position, background, object size, partial obstruction and relevant product differences. Include ordinary imperfect examples, not only deliberately extreme ones. If an unexpected condition is outside the intended scope, record how the system will recognise or handle that limitation.
For the bracket example, photographs from several production days may be more revealing than hundreds of near-identical frames from one short session. Repeated images of the same object can make a dataset look diverse while containing little new information.
Protect the test from the training process
Keep evaluation examples separate from material used to build or tune the model. Pay attention to related images: adjacent video frames or multiple photographs of the same item can create leakage even when the filenames differ. Document the split so a later result can be interpreted honestly.
If you repeatedly change the model after inspecting the same test errors, that test is becoming part of development. Reserve an appropriate independent evaluation for the final decision, with help from someone experienced in the task when the stakes justify it.
Review errors by consequence
In our hypothetical sorting process, sending a good item for manual review costs time; allowing a defective item through may have a different consequence. Report these outcomes separately instead of hiding them inside one average. Also inspect performance by the conditions you listed at the start.
Create a small error log with the image reference, expected answer, output and likely context. Avoid collecting unnecessary personal or confidential imagery. The log should make it possible to ask a concrete question: did the model struggle with glare, an unseen product version or an ambiguous label?
A test set is a description of the world the system was evaluated against. Keep that description beside the result. When the camera, product or environment changes, you then know which assumptions need another look.
Continue exploring: A Useful AI Summary Needs a Checking Routine, Not Just a Better Prompt.