A model can pass validation at release time and still become less useful later. Customer behavior changes, sensors drift, product catalogs evolve, fraud tactics adapt, upstream pipelines change, and the relationship between features and outcomes can weaken. Model monitoring exists to detect these changes early enough that the team can investigate before model quality silently degrades.

Google Cloud currently provides Model Monitoring v1 and v2. v1 is generally available and is configured around models deployed to Vertex AI endpoints; v2 is the newer preview offering that associates monitoring with a model version and can support broader monitoring scenarios. For the Professional Machine Learning Engineer, the central skill is interpreting monitoring signals rather than treating every statistical alert as an automatic retraining command. Google now calls the broader Vertex AI Platform the Gemini Enterprise Agent Platform, while current monitoring documentation continues to use the Vertex AI Model Monitoring feature name and version labels discussed here.

Monitoring starts with a baseline and a reason to compare

A monitoring metric is meaningful only relative to a baseline. Training-serving skew compares production feature distributions with the data used to train the model. Drift compares a newer production distribution with a previous production baseline or another reference. Output drift can compare the distribution of model predictions, while attribution monitoring can track whether the model’s reliance on features changes over time.

The baseline should represent the behavior the team considers healthy. If the training set was already biased, stale, or unrepresentative, skew detection can faithfully report that production differs from a bad reference. Monitoring does not validate the quality of the baseline itself.

Document why each baseline exists and when it should be refreshed. A seasonal business may expect December traffic to differ from July; a fraud model may need a rolling production baseline because adversarial behavior evolves continuously.

Training-serving skew reveals feature distribution mismatch

Skew occurs when the feature values reaching the model in production differ materially from the feature distributions in the training data. This can happen because the world changed, but it can also reveal an engineering defect such as different preprocessing code, missing values, unit conversion, category mapping, or data source selection.

That distinction matters operationally. Retraining on the original pipeline will not fix a serving bug that multiplies a feature by 1000. When skew alerts, first compare the feature-generation logic and raw inputs across training and serving. Confirm that schema, transformations, defaults, and feature ordering are consistent.

Feature skew is therefore both a data-quality signal and a model-risk signal. It says the model is operating outside the distribution used to justify its release, but it does not tell you why.

Drift shows that production data is changing over time

Inference drift tracks how feature distributions evolve in production. A retail demand model may see product mix change after a launch. A credit-risk model may see macroeconomic variables shift. A churn model may receive customers from a new acquisition channel whose behavior differs from historical users.

Some drift is normal and harmless. A feature can move while model accuracy remains stable, especially when the model is insensitive to that range or another feature captures the relevant signal. Other drift can be an early warning before labels arrive and direct performance metrics are available.

Use drift as a prompt for analysis. Determine which features changed, whether they are important, whether the change is expected, and whether labeled outcomes confirm a performance impact.

Model Monitoring v1 and v2 have different operational roles

Model Monitoring v1 is generally available and focuses on endpoint-based monitoring for feature skew and drift. It can sample prediction inputs, calculate distribution distances, apply feature thresholds, and generate alerts when configured limits are exceeded. This is appropriate for production support when the model is deployed to a supported Vertex AI endpoint.

Model Monitoring v2 is in preview and centralizes monitoring around a model version. It can run monitoring jobs on demand or on a schedule and supports objectives such as input-feature drift, output-inference drift, and feature-attribution change. It can also accommodate monitoring models that are served outside the traditional Vertex AI endpoint path.

Because v2 is preview, production-support requirements matter. Teams should not migrate merely because the newer version has broader capabilities. Google documents that v1 can continue to be used and that both versions can coexist during migration to avoid monitoring gaps.

Thresholds should reflect risk, not aesthetic neatness

Statistical distance metrics produce numbers, but the threshold that creates an alert is a business decision. A very low threshold can create constant noise from harmless variation. A very high threshold can delay detection until the model is operating in a substantially different population.

Start with historical data and replay known periods of normal variation. Measure how the distance behaves around holidays, promotions, regional launches, and other expected changes. Then compare periods where model quality actually degraded. A useful threshold separates actionable changes from normal background movement as much as practical.

Critical features may deserve tighter thresholds than weak features. A feature whose change directly affects a regulated decision can warrant immediate review even if another feature can drift widely without changing outcomes.

Feature attribution change can reveal a different decision mechanism

Feature attribution describes how much features contribute to a model’s prediction for a given instance. Monitoring attribution over time can reveal that a model is making similar predictions for different reasons. For example, a previously dominant behavioral feature might lose influence while a demographic proxy becomes more important.

This signal can be valuable even before aggregate accuracy changes. A shift in the model’s decision mechanism can reveal data-pipeline changes, new correlations, unstable features, or emerging fairness concerns. It should trigger deeper model and data analysis rather than a simple “retrain” response.

Attribution monitoring is especially useful for models where explainability is part of governance, because it connects operational change to the evidence reviewers use to understand model behavior.

Performance monitoring needs labels, not just feature statistics

Distribution monitoring is a proxy for model risk. The most direct evidence of degradation is a change in task performance measured against ground truth: accuracy, precision, recall, calibration, ranking quality, forecast error, business loss, or another metric tied to the decision.

Ground truth often arrives late. Fraud chargebacks, loan defaults, customer churn, and medical outcomes can take days or months to become known. That is why teams use feature and prediction drift as early signals while maintaining a delayed performance-evaluation pipeline when labels arrive.

Do not declare a model healthy solely because no skew alert fired. A stable input distribution can still produce worse outcomes if the relationship between features and labels changed—a form of concept drift that feature statistics may not detect directly.

Retraining should be triggered by evidence and policy

An alert can start a retraining workflow, but it should not automatically promote the resulting model. The pipeline should diagnose data quality, rebuild the candidate with current data when appropriate, evaluate it against the production version, and apply the same release gates used for any other model change.

Some alerts require no retraining. A broken upstream feature pipeline should be fixed. A planned product launch may justify a temporary threshold adjustment. A newly important subgroup may require collecting labels before a responsible model update is possible.

This is why the MLOps lifecycle connects monitoring, data engineering, evaluation, and deployment rather than treating monitoring as a standalone dashboard.

Sampling and monitoring frequency are cost-and-detection tradeoffs

Monitoring every prediction can be expensive for high-volume services, while monitoring too small a sample can miss changes in rare but important segments. Choose a sampling rate that provides enough observations for stable feature distributions and consider whether high-risk traffic needs separate monitoring.

Monitoring frequency determines how quickly a change can be detected and how much data is included in each window. A fast-moving fraud system may need shorter windows than a monthly forecasting model. Extremely short windows can also make natural randomness look like drift.

Align alerting with the operational response. There is little value in hourly alerts if the team can investigate only once per week, and there is little value in weekly monitoring if a harmful model can make thousands of irreversible decisions in a few hours.

Keep alert windows comparable when investigating a trend. Changes in sampling fraction, traffic routing, or monitoring cadence can alter the observed distribution and create apparent drift that is really a measurement change.

Monitoring design should also define ownership for every alert class. Data engineers may own freshness and schema failures, ML engineers may own skew and attribution investigation, and product or risk teams may own business-performance or fairness thresholds. Routing all alerts to one generic queue creates delay and encourages broad threshold increases just to reduce noise.

Model quality depends on data freshness, feature jobs, endpoint health, latency, error rate, resource saturation, and downstream business logic. A model can be statistically healthy but operationally unavailable, or it can serve predictions quickly while receiving stale features.

Combine model-monitoring signals with pipeline freshness, schema validation, prediction-service metrics, application SLOs, and business KPIs. Link every alert to a runbook that identifies the likely owners and evidence sources.

The purpose of monitoring is not to prove that a model never changes. It is to make change visible and interpretable. When teams can distinguish harmless drift from broken features and genuine performance decay, retraining becomes a controlled engineering decision instead of a reflex triggered by a chart.