AI Agent Observability: Tracing, Metrics, and Evals in Production

Why agents need step-level tracing, not just logs

The classic three pillars of observability (logs, metrics, traces) all apply to agents, but the weighting changes. With a normal web service, logs do most of the heavy lifting. With agents, the trace is the center of gravity, because the interesting information lives in the relationships between steps, not in any single line.

Consider the shape of a real run. A research agent asked to compile a competitive summary might run for several minutes and emit hundreds of spans: model calls, tool calls, sub-agent invocations, retries after rate limits. If you only kept flat logs, you would have a thousand disconnected lines and no way to know that span #412 (a tool error) is what forced the model into the wrong conclusion at span #889. A trace stitches those together by parent-child relationships, so the causal chain is visible at a glance.

Step-level tracing also makes nondeterminism manageable. The same prompt can produce different paths on different days. Without per-step capture you cannot tell whether yesterday's good run and today's bad run differed because of the model, the data, the tools, or a prompt change. With it, you diff two traces side by side and the divergence point is obvious.

There is a practical performance note here too. The instrumentation overhead of capturing spans is tiny (well under a millisecond per call) compared with the model latency that dominates an agent run, which can range from a hundred milliseconds to tens of seconds per call. You are not paying a real speed tax to see what your agent is doing.

The four pillars: traces, metrics, logs, and evals

Good agent observability rests on four kinds of signal. Each answers a different question, and you want all four because any one alone leaves a blind spot.

Here is how they divide up the work, with the question each one is built to answer:

• Traces answer "what happened, in what order, and why." A trace is the full tree of one run: every span (a model call, a tool call, a retrieval) nested under its parent, with timing, inputs, and outputs attached. This is your primary debugging surface.
• Metrics answer "how is the system behaving in aggregate." These are the numbers you chart over time: p50 and p95 latency, tokens and dollars per run, tool error rate, task success rate, and run volume. Metrics tell you a regression happened; traces tell you which run to open.
• Logs answer "what did this specific component say." Structured logs (not free-text strings) capture events inside a step: a tool returned a 429, a guardrail blocked an output, a retry fired. Attach them to the span so they show up in context, not in a separate haystack.
• Evals answer "was the output actually good." An eval scores quality (was the answer correct, grounded, on-policy) rather than whether the run technically completed. Offline evals run against a fixed test set before you ship; online evals score live production traces so quality regressions surface in production, not in a user complaint a week later.
• Together these form a loop: metrics flag a problem, traces locate it, logs explain the failing step, and evals tell you whether your fix actually improved quality rather than just changing the output.

What to capture at each step

The temptation is to capture everything or to capture almost nothing. Both are mistakes. Capture too little and a trace becomes useless the moment you actually need it; capture raw everything and you create a privacy liability and a storage bill. The right answer is to capture the fields that let you reconstruct a decision, while masking sensitive values.

For each span, aim to record the following. Each item earns its place because it is the thing you will wish you had logged at 2am during an incident:

• The step type and name: was this a model call, a tool call, or a retrieval, and which specific tool or model. Without this you cannot even read the trace.
• Inputs and outputs: the prompt or tool arguments going in, and the response coming back. This is the single most valuable field because it lets you see what the agent actually saw, not what you assumed it saw.
• Timing and token cost: start time, duration, input and output tokens, and the dollar cost of the call. This is what turns a trace into a budget and latency record, not just a debug dump.
• The decision rationale where available: which tool the model chose and, if exposed, why. Tool-call selection is where agents most often go off the rails, so capturing the chosen tool plus arguments is essential.
• Status and errors: success, failure, timeout, retry, or guardrail block, with the error payload. A trace that hides its failures is worse than no trace.
• Identity and context: which user or tenant initiated the run, which agent version and prompt version ran, and a correlation ID. Mask PII and secrets at capture time so the trace is safe to keep.

How agent observability works, step by step

The mechanics are less mysterious than they sound. Most agent observability follows the same pipeline whether you build it yourself or use a platform. The current convention is to instrument against the OpenTelemetry GenAI semantic conventions rather than a single vendor's proprietary SDK, so your trace data stays portable and you are not locked into one backend.

The detail that trips people up is context propagation. If a retry or a sub-agent starts its own trace instead of attaching to the parent, you end up with orphan fragments and lose the causal chain. Getting propagation right is what makes a deep, multi-step run readable as a single story.

Once the trace lands, the value compounds. A production failure can be promoted into an eval case with one action, so the exact scenario that broke today becomes a regression test that protects you tomorrow. That feedback loop is the whole point: observability is not just watching, it is learning.

The metrics that actually matter for agents

Plenty of dashboards track vanity numbers. For agents, a short list of metrics carries most of the signal. The chart below shows a realistic split of where time and money go on a typical multi-step agent run, which explains why latency and cost optimization usually starts with the model and retrieval steps rather than the tool calls.

Watch latency at the p95, not just the average, because agents have long tails: a run that retries twice or calls a slow tool will blow past your median. Track cost per run, not just cost per token, since a chatty agent that makes ten model calls can be expensive even on a cheap model. And treat task success rate (did the run accomplish the goal) as separate from completion rate (did the run finish without crashing). A run can complete cleanly and still be wrong.

A worked example: debugging a wrong answer

Make it concrete. A support agent at a logistics company is asked by a rep: "What is the refund status for order 88142?" The agent replies that the order was refunded on May 2. The customer says they never got a refund. The rep escalates. Without observability, an engineer is now reverse-engineering a black box from a single bad sentence.

With step-level tracing, the engineer opens the run and reads the tree. Span 1: the model parses the request and decides to call a lookup tool. Span 2: the tool call goes out with order_id 88412 (a transposition). Span 3: the tool returns a real record for that wrong order, which happens to be refunded. Span 4: the model, having no reason to doubt a successful tool response, reports the refund. The final answer was confident and completely wrong, and the trace shows exactly why in under a minute.

The fix is now obvious and testable. The model transposed digits, so the team adds a validation step that echoes the parsed order ID back for confirmation on high-stakes lookups, and they turn this exact run into an eval case. Next time the agent transposes a digit, the eval catches it before a customer does. That is the entire value of observability in one story: a vague complaint became a precise, fixable, regression-tested bug.

How observability connects to audit logs and governance

Observability and audit logging look similar (both record what an agent did) but they serve different masters, and conflating them is a mistake. Observability is built for engineers debugging behavior; it is detailed, mutable, sampled, and often short-lived. An audit log is built for compliance and security; it is immutable, retained on a schedule, and answers "who did what, when, and under what authority" for a reviewer who may read it months later.

The good news is they share a source. The same instrumented run that produces an engineering trace can emit the durable, tamper-evident records an audit log needs: which user triggered the agent, which action it took on which system, whether a human approved it, and the result. When an agent updates a Salesforce deal or resolves a Zendesk ticket, the audit log is the proof that the action was authorized and reviewable. Observability tells you why the agent did it; the audit log proves that it was allowed to.

This is where governance enters. Least-privilege access controls decide what an agent is permitted to touch, human-in-the-loop approvals gate the risky actions, and audit logs record the outcome. Observability sits underneath all three, because you cannot govern what you cannot see. A platform like Onpilot is built around exactly this pairing: agents take governed action across connected systems with approvals and least-privilege roles, and every step is captured so the audit trail and the engineering trace come from the same source of truth.

If you are formalizing this, the same trace-and-log discipline maps cleanly onto established frameworks for AI risk, and it is the backbone of any serious answer to a SOC 2 auditor about your agents.

Observability vs related practices

Teams often confuse observability with monitoring, with audit logging, or with evaluation. They overlap but are not interchangeable. The table below lays out what each one is for, so you can tell which gap you are actually filling.

The practical takeaway: monitoring catches outages, observability explains them, evals tell you about quality the metrics miss, and audit logs satisfy the people who do not care about your dashboards but very much care about accountability. Skipping any one of them leaves a gap someone will eventually fall through.

Common mistakes that make traces useless

Most observability failures are not about missing tools. They are about capturing the wrong things, or capturing the right things in a way that makes them unusable. These are the patterns that quietly defeat teams:

• Logging without trace context, so you have a million lines and no way to connect them into a single run. Flat logs are nearly worthless for debugging a multi-step agent.
• Capturing inputs and outputs raw, including PII and secrets, which turns your trace store into a compliance liability. Mask sensitive fields at capture time, not after the fact.
• Tracking completion rate but not task success, so your dashboard is green while the agent is confidently wrong. A run that finishes is not a run that succeeded.
• Breaking context propagation on retries and sub-agents, which scatters one logical run into orphan fragments and hides the causal chain you most need to see.
• Treating evals as a pre-launch gate only. Without online evals on production traces, quality drift goes unnoticed until a customer complains, which is the most expensive way to learn.
• Over-instrumenting everything at full fidelity with no sampling, which buries the signal and runs up storage costs until someone disables it entirely. Keep full detail on errors and a sample of the rest.

A decision framework: how much observability do you need

Not every agent needs the same depth. The right level scales with blast radius (how much damage a bad run can do) and with how often the agent runs unattended. Use this as a rough guide rather than a rule.

Use lightweight tracing plus basic metrics when the agent is read-only and a human reviews every output before anything happens. A drafting agent that suggests email replies a person sends manually is low blast radius; full traces on errors and aggregate latency and cost metrics are usually enough.

Use full step-level tracing, online evals, and audit logging when the agent takes action on systems of record, runs on a schedule without a human watching each run, or touches regulated data. The moment an agent can update a CRM record, resolve a ticket, or move money, you need to be able to replay any run and prove what happened. This is also the threshold where human-in-the-loop approvals and least-privilege roles stop being optional.

Scale evals to the cost of being wrong. A marketing summary that is occasionally bland is cheap to get wrong; a finance agent that miscategorizes a transaction is not. Heavier eval coverage (more cases, stricter scorers, online checks) belongs wherever a wrong answer is expensive or hard to reverse. The honest rule of thumb: instrument to the level where, if a regulator or an angry customer asked you to explain a specific run, you could pull it up and walk them through it line by line.

Getting started without boiling the ocean

You do not need a perfect observability stack on day one. You need to be able to open a failed run and read it. Start there and expand.

Begin by instrumenting the spine of a single agent: a root span per run, child spans for each model and tool call, with inputs, outputs, timing, and tokens. Mask PII as you capture. That alone will solve most debugging pain. Add metrics next (latency percentiles, cost per run, task success) so you know which runs to open. Then layer in evals, starting with a handful of cases drawn from real failures you have already seen. Wire online evals onto production traces once the offline set is stable.

Lean on the OpenTelemetry GenAI conventions so your data is portable and you can change backends without re-instrumenting. And connect the same capture to your audit log early, because retrofitting durable, authority-aware records onto an observability stack built only for debugging is far more work than doing it together from the start.

The payoff is compounding. Every failure you trace becomes an eval case, every eval case makes the next regression cheaper to catch, and the same data that helps your engineers also satisfies your auditors. Observability is the unglamorous foundation that lets you run agents in production and actually sleep at night.