Anomaly detection: Catch the defects you didn’t train for

Written by Tobias Skov
Machine vision

Traditional machine vision systems are very good at finding the defects they have been trained to detect. But what happens when something unexpected appears?

If a defect falls outside the training data, there is a real risk that the system will simply pass it. A human inspector, on the other hand, would often react immediately because something just looks wrong.

That intuitive reaction is exactly what anomaly detection brings to machine vision: a layer of common sense.

How anomaly detection works

In a traditional supervised setup, a vision system is trained on examples of specific defect types. It learns to recognize scratches, cracks, contamination, or other predefined issues.

An anomaly detection model works differently.

Instead of learning what defects look like, it learns what normal looks like. The model is trained on good items. By understanding the typical shape, texture, color, and structure of a normal product, it can flag anything that deviates significantly from that baseline.

anomaly detectionOn the left are examples of good items, on which the anomaly detection algorithm is trained. On the right are the inspected items, where the two marked in red are flagged as deviations.

Easier to build a training set

One of the biggest challenges in AI-based quality control is collecting enough defect images for training. In many production lines, defects are rare. This is, of course, good for production, but problematic for data collection.

Anomaly detection solves this issue. Since it is trained on good items, and production lines always produce far more good items than bad ones, building a training dataset becomes significantly easier. In addition, anomaly detection models often require fewer images to achieve robust performance.

This makes deployment faster and more realistic in everyday manufacturing environments.

Different ways to use anomaly detection

Anomaly detection can be deployed in several ways, depending on the application.

1. First line of defense

In some setups, anomaly detection acts as an initial filter. It determines whether an item looks normal or abnormal. If it flags an item as defective, a supervised model can then classify the specific defect type.

This layered approach combines flexibility with precision.

2. Last line of defense (freak detection)

In other cases, anomaly detection serves as a safety net, or what we call freak detection.

These are defects that are so rare that you haven’t predicted them or didn’t have enough data to train a supervised model. They may fall completely outside your defined defect categories.

Here, anomaly detection provides an extra layer of protection, ensuring unexpected issues don’t slip through unnoticed.

Applications in complex and organic materials

In recent years, anomaly detection has matured significantly. The accuracy levels now make it viable even for demanding inspection tasks, including aesthetic inspection of organic materials such as wood.

Organic materials pose a particular challenge because their natural variation is wide. Defining every possible defect explicitly can be nearly impossible. Anomaly detection handles this complexity by learning the natural variation of acceptable products and flagging deviations beyond that range.

Powerful but no silver bullet

Anomaly detection is powerful, but it’s not a silver bullet.

The output is typically binary: normal or defective. The model does not classify the defect or explain what is wrong. Threshold tuning is also critical. Set the threshold too tight, and you risk false rejects. Set it too loose, and defects may pass.

Anomaly detection does not replace traditional machine vision or supervised AI models. It complements them.

By adding a “common sense” layer,  you reduce the risk of unknown defects slipping through and make your inspection system more robust against the unexpected.

In industrial quality control, that extra layer can be the difference between a system that performs well in theory and one that performs reliably in real production.