What Is Model Monitoring? What to Track After AI Goes Live

ProductionObservabilityPublished Updated By Simon Budziak

Model monitoring tracks a deployed AI model's inputs, outputs, quality, speed and cost to catch decay before users do. It watches for data drift, falling accuracy, rising errors and unusual behavior, and raises an alert when a threshold you set is crossed.

What should you monitor on a live model?

Signal Example Why it matters
Input data Share of missing fields, new categories, shifted ranges Early sign of drift
Output quality Accuracy against labeled samples, judge scores, override rate The thing you actually care about
Operations Latency percentiles, error and timeout rate, throughput User-visible reliability
Cost Tokens or compute per task Spend can creep without any error
Safety Refusals, blocked outputs, policy hits Misuse and failure modes

Quality is the signal that lags. True labels often arrive days or weeks later, so teams watch input drift and proxy signals, such as how often a person edits an output, as early warnings.

How is model monitoring different from LLM observability?

Monitoring answers “is it still within bounds?” and alerts. Observability answers “why did this one fail?” by keeping traces of every step. They share data, so most teams log traces first and build monitors on top. See LLM observability for the tracing side.

How do we monitor the agents we run?

Our own scheduled agents write and ship this site, several runs a week, so we assume a model may weaken a safeguard it can edit. In our post on agents that disabled their own monitoring we describe the controls that hold: every run works on a branch and opens a pull request, a review gate and the site’s checks run in CI, and a separate merge workflow merges only when every check is green. The writing agent has no merge rights. We kill and replace sub-agents that go silent past a time ceiling, and we check a run’s claims against the artifacts it produced rather than its own summary. A monitor the monitored system can switch off is a setting, not a control.

How do you set alerts without drowning in noise?

Alert on thresholds tied to a decision, such as “override rate above 20% for two days means pause the automation”, not on every wobble. Use rolling windows for statistics, set owners for each alert and review thresholds after each incident. Every alert should name the action it triggers. Add guardrails for hard limits, and use anomaly detection for signals without a natural threshold. Pair monitoring with a rollback path, described under model deployment.

Written with AI assistance and reviewed by Simon Budziak. The production notes come from systems Soba Labs builds and runs.

Frequently asked questions

What is the difference between model monitoring and observability?

Monitoring watches known signals against thresholds and alerts you. Observability keeps enough traces and context to investigate failures nobody predicted. A live system needs both.

How often should a model be re-evaluated?

Monitor continuously, and re-run a fixed evaluation set on a schedule and after every change to the model, prompts, tools or data feeds.

Summarize this page with

Train your team to build this