Picture a dashboard. Error rate zero. Latency comfortably inside SLO. Throughput steady, no saturation, no restarts, uptime four nines. Every signal green.
And the agent behind it has been coding invoices to the wrong expense account for eleven days.
Nothing on that dashboard is broken or misconfigured. The dashboard is answering the question it was built to answer, correctly. The problem is that it was built for a different kind of system.
Application observability tells you whether the system is working. Agent observability has to tell you whether it is right. Those are not the same question, and the first one has better tooling.
This piece covers why the standard observability model has no slot for correctness, the four things agent monitoring actually needs to watch, the ground truth problem that makes this genuinely hard, and why testing is what makes observability possible rather than an alternative to it.
Why the Three Pillars Don’t Cover This
Metrics, logs and traces are a genuinely good model, and they were designed against a specific assumption: that a system is behaving correctly when it is not failing. For deterministic software that assumption mostly holds. If the code is right and nothing errored, the output is right.
Agents break the assumption. An agent can execute flawlessly and decide wrongly, and no pillar has anywhere to record that.
| Pillar | Answers | Cannot Answer |
| Metrics | How many, how fast, how often | Whether the decisions counted were correct |
| Logs | What the system did | Whether it should have done that |
| Traces | The path a request took through the system | Whether the destination was the right one |
You can log every decision an agent makes, in full detail, with perfect trace context, and still have no idea whether any of them were right. The log records the action. Correctness is a property of the relationship between the action and what the situation called for, and that relationship is not in the telemetry.
Which is why agent observability needs something the three pillars do not provide: a continuous comparison between what the agent decided and what should have been decided.
Four Things Agent Monitoring Has to Watch
Application observability signals, latency, errors, saturation, remain necessary and are well covered by existing tooling. These four are the additions.
1. Decision quality
The proportion of agent decisions that are correct against your business rules, measured continuously rather than sampled. This is the primary signal and the hardest to produce, for reasons covered in the next section.
2. Scope adherence
Whether the agent is acting inside its authorized boundary, value thresholds, entity or organizational restrictions, transaction types it may touch. Unlike decision quality, this is comparatively easy to monitor, because the boundary is explicitly defined somewhere and each action can be checked against it.
The signal to watch is not only breaches but the distribution of how close actions run to the boundary. An agent that begins clustering just under an approval threshold is doing something worth understanding, even though no individual action violated anything.
3. Behavioural drift
Whether the agent’s decision distribution has moved from the validated baseline the five validation dimensions. An agent going from 94% to 87% correct coding produces no errors and no failed transactions, it simply degrades, and only a distribution comparison surfaces it.
4. Evidence completeness
Whether every agent action is producing a retrievable record of what was decided, on what input, under which rule. This is the one most likely to be discovered missing during an audit SOX evidence requirements, rather than during operations, and it is worth monitoring as a first-class signal precisely because nothing else will surface its absence.
The Ground Truth Problem
Here is what makes decision quality genuinely difficult rather than merely unimplemented.
To know an agent decided wrongly, you need to know what the right decision was. In a test environment that is straightforward, you constructed the scenario, so you know the expected outcome. In production, for a decision the agent just made, you frequently do not.
Nobody independently coded that invoice. There is no parallel human doing the same work to compare against. The agent’s output is the only answer that exists, and asking whether it is correct is asking a question nothing in the system can currently answer.
In test, you know the right answer because you wrote the scenario. In production, the agent’s decision is often the only answer anyone has.
Three approaches get used, and each has a real limitation worth understanding before choosing:
- Rule-based validation. Check the decision against the business rules that should have governed it, did the account match the vendor’s coding rule, did the approval route match the delegation matrix. Strong where rules are explicit, and blind where the decision required judgment the rules do not encode.
- Downstream outcome. Wait for a later signal, the invoice was disputed, the entry was reversed at close, the customer complained. Genuinely reliable and arrives too late to prevent anything, which makes it useful for baseline calibration rather than alerting.
- Human sampling. Have someone review a subset. Provides real ground truth, does not scale, and, as covered below, samples are the wrong instrument for the thing you most need to detect.
The practical answer is usually all three at different cadences: rule-based validation continuously for the decisions rules can cover, downstream outcomes to recalibrate the baseline, and human review of a small stratified sample to check the other two have not drifted together.
Why Sampling Fails Here
Traditional monitoring samples because full instrumentation is expensive and, for most signals, a sample is statistically adequate. Latency at the 99th percentile from a 1% sample is a reasonable estimate.
For agent decision quality, sampling inverts from a reasonable economy into the wrong instrument, for a specific reason: **the thing you most need to detect is a property of the distribution, not of any instance. **
A shift from 94% to 87% correct is a seven-point move. Sampling fifty transactions gives you a confidence interval far wider than seven points, so the sample cannot distinguish the degraded state from the healthy one. You would need a sample large enough that you have effectively stopped sampling.
This is compounded by what a sample tends to contain. Randomly drawn transactions are mostly routine ones, because most transactions are routine. The decisions where an agent is likely to be wrong are the unusual ones, which are, by definition, underrepresented in a random sample.
You cannot sample your way to detecting a seven-point quality shift. By the time it is visible in a sample, it has been true for months.
Where sampling does work is human review as a calibration check on automated validation, a small stratified sample, weighted toward unusual cases, asking whether the automated checks are still measuring what they should. That is a different job from detecting drift, and it is the job sampling is actually suited to.
Alerting on Something That Isn’t an Error
The alerting model most teams run is threshold-based on error signals: error rate exceeds X, latency exceeds Y, queue depth exceeds Z. Every one of those thresholds is built on the assumption that a problem produces a failure.
Agent quality degradation produces no failures, so those alerts never fire. Which means agent observability needs alerting on distribution shifts, a fundamentally different model, and one most monitoring stacks are not configured for.
| Signal | Traditional Alert | Agent Alert |
| Decision quality | No equivalent | Correct-rate falls below a defined band |
| Scope adherence | No equivalent | Any breach; also clustering near a threshold |
| Distribution shift | No equivalent | Account or route distribution moves beyond variance |
| Volume | Throughput anomaly | Sudden change in decisions per period, could indicate scope expansion |
| Escalation rate | No equivalent | Escalations drop, an agent that stopped escalating is not more confident, it is less cautious |
The last row is worth dwelling on because it is counterintuitive and rarely instrumented. A falling escalation rate looks like improvement on any dashboard, fewer exceptions, less human involvement, smoother operation. It can equally mean the agent has started handling cases it should be escalating, which is exactly the failure mode you would most want to catch early.
Testing Is What Makes Observability Possible
These two practices get discussed as separate disciplines, and for agents they are not separable in one specific direction: **drift detection requires a baseline, and the baseline comes from validation.**
If you never established what correct looked like for this agent, in this configuration, on this version, you have nothing to compare production behavior against. You can observe that the agent codes 71% of invoices to account 6400. You cannot say whether that is normal, because normal was never recorded.
Which produces a sequencing requirement that is easy to get wrong. Baselines have to be captured at deployment, when the agent has been validated and its behavior is known-good. Reconstructing a baseline six months later from production data is not really possible, whatever the agent is doing now becomes your definition of normal, including whatever drift has already happened.
Capture the baseline at deployment. Reconstructing it later just enshrines the drift you were trying to detect.
The same relationship runs the other way after any legitimate change. When an agent is updated, or a business rule changes, the old baseline is invalid and a new one has to be established through validation, otherwise every subsequent comparison is against a state that no longer applies.
Where Sofy Adds Value
Sofy Failure Analysis Agents sits on the validation side of that relationship, which is the side that makes the observability side work.
Continuous validation of agent outcomes at the data layer produces two things observability needs: the baseline itself, the recorded distribution of correct behavior at a known-good point, and an ongoing comparison against it, run against every transaction rather than a sample. That addresses the decision quality signal and the drift signal directly, which are the two hardest of the four.
For scope adherence, the same validation checks whether actions fell inside agents authorized boundaries, including the near-threshold clustering that no breach alert would catch. And because each validation produces a timestamped record, evidence completeness stops being a separate instrumentation problem, the record is a by-product of the check rather than something to build alongside it.
What it does not do is replace your APM stack. Latency, errors, saturation and infrastructure health are well served by existing tooling and there is no reason to move them. The addition is the correctness layer those tools were never designed to carry.
What This Doesn’t Solve
Three honest limits.
- Rules cannot cover judgment. Rule-based validation works where correctness is defined by an explicit rule. Where an agent is exercising judgment the rules do not encode, automated correctness checking has a ceiling, and human review of unusual cases remains the only real instrument.
- Observability detects; it does not decide. Knowing that decision quality has fallen four points tells you something changed. Whether that warrants pausing the agent, narrowing its scope or accepting the new level is a business risk decision, and the data informs it rather than making it.
- A baseline is only as good as the validation behind it. If the deployment-time validation was thin, the baseline encodes an invalidated state and every subsequent comparison inherits that. Drift detection against a bad baseline produces confident, wrong reassurance.
Frequently Asked Questions
What is agent observability?
Agent observability is monitoring whether AI agents in production are making correct decisions, staying within their authorized scope, behaving consistently with a validated baseline, and producing retrievable evidence. It sits alongside application observability rather than replacing it, metrics, logs and traces still tell you whether the system is running, and none of them can tell you whether an agent’s decisions are right.
Why isn’t standard APM enough for monitoring AI agents?
Because the three pillars assume correct behavior is the absence of failure. Metrics count actions without assessing them, logs record what happened rather than whether it should have, and traces show the path rather than whether the destination was right. An agent can run with zero errors and perfect latency while making systematically wrong decisions, and no pillar has a place to record that.
How do you know whether an agent decision was correct in production?
Three approaches, usually combined. Rule-based validation compares the decision against the business rules that should have governed it, continuous, but blind where judgment was required. Downstream outcomes (disputes, reversals, complaints) are reliable but arrive too late to prevent anything, so they suit baseline recalibration. Human review of a small stratified sample provides real ground truth and checks the other two have not drifted together.
Why can’t you sample agent decisions for quality monitoring?
Because the thing you most need to detect is a property of the distribution rather than of any instance. A shift from 94% to 87% correct is seven points, and a fifty-transaction sample carries a confidence interval far wider than that, so the sample cannot distinguish degraded from healthy. Random samples also skew toward routine transactions, while the decisions an agent is most likely to get wrong are the unusual ones.
What should trigger an alert for an AI agent?
Distribution shifts rather than error thresholds, since agent quality degradation produces no errors. Useful triggers include correct-rate falling below a defined band, any scope breach plus clustering near a threshold, account or routing distributions moving beyond expected variance, and, counterintuitively, a falling escalation rate, which looks like improvement but can mean the agent has started handling cases it should be escalating.
The Bottom Line
Testing establishes that an agent behaves correctly before it is deployed. Observability is how you know it still does at month eight, and the two are more tightly coupled than the usual framing suggests, you cannot detect drift from a baseline you never captured.
The harder point is that the observability tooling most organizations already run cannot carry this. It was built on an assumption that no longer holds: that a healthy system is a correct one. An agent breaks that assumption cleanly, and every green dashboard built on it inherits the blind spot.
The addition is a correctness layer, continuous comparison of what the agent decided against what should have been decided, alerting on distribution rather than error, and a baseline captured at deployment while you still know what good looks like.
Establish the Baseline Before You Need It
See continuous outcome validation produce the known-good baseline that makes drift detection possible, and the per-transaction comparison that keeps it honest.