AI is increasingly used in quality control, but for quality control in the pharmaceutical industry and medical device inspection, the standard approach often falls short. That’s because most AI or machine learning models are black boxes. You get the right result, but you can’t always explain how or why.
In the medical and pharmaceutical industry, where patient safety is at stake and FDA approvals are needed, that isn't acceptable.
However, at JLI, we’ve managed to tackle the challenge and develop a validated AI solution for quality control in this industry. Here’s how we make it work.
Full traceability from training data to final model
The foundation of any AI solution is the training data. But for pharma, it’s not just about having enough images, it’s about documenting everything:
- Where each image came from
- How it was annotated
- How it was used during training
We use supervised learning and train on images we have annotated ourselves. These images are then split into training, validation, and test sets, and we maintain that separation throughout the project.
If new data is added later, it’s kept separate and only split internally, never merged with the existing dataset. Reusing an old split or mixing datasets could introduce bias and invalidate the model, so we avoid any form of “cross-contamination”.
Documenting the training process
We track every aspect of the model training using Weights & Biases (W&B), a platform that, among other things, logs:
- Annotation changes
- Model parameters
- Hyperparameter tuning
- Version history
This gives us the ability to retrace the entire development process and show exactly how the final model came to be. With this audit trail, every change is documented and every result is reproducible.
Locking the model for validation
Once the model reaches the desired performance, we lock it and calculate a checksum to ensure the deployed version can always be verified against the approved version. If the checksum changes, you know the model has been modified.
This is essential for compliance in regulated industries like pharma.
Input monitoring to ensure consistency
Even the best model can fail if it’s exposed to images that differ too much from the training data. That’s why we implement input monitoring, which tracks incoming images to ensure they stay within the expected range.
This is particularly important when inspecting materials that may vary over time, like organic substances.
If the input begins to drift, the system can raise a flag so corrective actions can be taken before inspection accuracy is compromised.
Extreme precision: Going from 98% to 99.9%
In pharmaceutical inspections, near-perfect isn’t good enough. Models need to be optimized not just for accuracy, but for reliability across all defect types.
Achieving that last push, from 98% to 99.9% accuracy, is a bit of an engineering feat and requires hours of iterative refinement: rebalancing datasets, retraining with rare error cases, and fine-tuning parameters.
But when you get there, with full traceability, locked models, and monitored inputs, you have an AI-based machine vision solution that can be validated and used for critical inspection tasks in the medical industry.
