What Exactly Is a “Model” – and Why Does It Degrade Over Time?
A model that performed brilliantly at validation can quietly become unreliable a year later without anyone changing a single line of code. This phenomenon has a name – drift – and understanding it is essential for anyone operating AI in production.
First, what a model actually is: the output of the training process. Concretely, it is a file containing an architecture (the structure of the computation) and parameters (millions of numbers tuned during training). Given an input – an image, a sequence of events, a sensor window – it produces an output: a classification, a score, a prediction. The model is frozen at training time; it embodies the world as represented by its training data.
And there lies the problem: the world does not stay frozen.
Data drift happens when the inputs change: a new label supplier with slightly different ink, a replaced camera, a new market with different aggregation practices, seasonal shifts in logistics flows. The model receives inputs statistically different from anything it trained on – and its accuracy erodes.
Concept drift is subtler: the relationship between input and answer changes. A pattern that used to indicate diversion becomes normal after a distribution network redesign. Yesterday’s anomaly is today’s routine.
The insidious part is that ML models rarely fail loudly. They keep producing outputs – confident, plausible, increasingly wrong. Without monitoring, nobody notices until the consequences surface downstream.
The countermeasures are organizational as much as technical:
- Monitor in production. Track input distributions and output statistics continuously; alert when they shift from the training baseline.
- Keep a feedback loop. Human-confirmed outcomes – true defect or false reject, real anomaly or explained event – are gold: they measure real accuracy and become tomorrow’s training data.
- Plan retraining as a lifecycle, not an emergency. Versioned datasets, documented retraining triggers, revalidation protocols. In regulated environments this is the heart of any credible AI governance.
The shift: a model is not a product you buy once. It is a living asset – closer to a calibrated instrument than to a software license – and it needs the same discipline of periodic verification.
