AI Observability & AgentOps

17.1 LLMOps/AgentOps & Tracing Agent Trajectories

Priority: Must Understand

Executive Definition: LLMOps/AgentOps is the operational discipline of running LLM and agent systems in production: capturing structured traces of every model call and tool invocation (inputs, outputs, latency, cost, intermediate reasoning steps), correlating them into a single end-to-end trajectory per user request, and using that trace data for debugging, evaluation, and cost attribution. It extends conventional software observability (logs, metrics, traces) with AI-specific concepts (prompts, completions, token usage, and multi-step agent trajectories) now being standardized as OpenTelemetry's GenAI semantic conventions.

Why It Matters: An agent failure or cost spike is undebuggable after the fact unless every step it took was captured at the time: you cannot re-derive what an agent "was thinking" or which tool arguments it used from application logs designed for conventional request/response services. The industry is actively converging on a shared standard for this (OpenTelemetry's GenAI semantic conventions define spans for LLM calls, tool invocations, and agent steps) specifically so that traces are portable across observability vendors instead of locked into a single platform's proprietary schema (opentelemetry.io).

What I Need to Understand:

• A trace for an agentic workflow needs to capture, at minimum: every prompt and completion, every tool call with its arguments and result, token counts and cost per step, latency per step, and the overall trajectory linking these into one causal chain: this is the raw data trajectory evaluation and incident response both depend on.
• OpenTelemetry's GenAI semantic conventions define standardized span types for LLM invocations and agent/tool steps, which matters for interoperability: traces captured this way can move between observability backends (Datadog, Arize, LangSmith, self-hosted) without a rewrite.
• Tracing has to be designed in from the start of an agent system's architecture: retrofitting full trajectory capture after an incident, when you need it most, is usually not possible.
• LLMOps/AgentOps tooling overlaps heavily with the eval tooling covered above: the same trace data that supports debugging is what a trajectory eval or a production canary comparison scores against.
• Distinguish platform-level observability (is the gateway/model healthy, latency, error rates) from trajectory-level observability (did this specific agent run behave correctly): both are needed, and they are not the same discipline.

Questions I Should Be Able to Ask My Team:

1. Can we reconstruct the full step-by-step trajectory (every tool call, argument, and intermediate output) for any agent run from the last 90 days, or only recent runs?
2. Are we using a standardized tracing format (OpenTelemetry GenAI conventions) or a proprietary schema tied to one vendor?
3. If our observability vendor changed tomorrow, would we lose historical trace data, or is it portable?

Technologies / Standards / Companies to Know: OpenTelemetry GenAI semantic conventions, LangSmith, Arize, Langfuse, Datadog LLM Observability.

Recommended Learning:

• Inside the LLM Call: GenAI Observability with OpenTelemetry: OpenTelemetry Blog: primary-source explanation of the emerging standard for AI tracing.
• OpenTelemetry GenAI Semantic Conventions: MLflow documentation: concrete documentation of what a compliant trace actually contains.
• AgentOps: Enabling Observability of LLM Agents (arXiv): academic framing of what agent observability needs to capture and why conventional Application Performance Monitoring (APM) falls short.

Time Investment: 2-3 hours

---

17.2 Incident Response & Audit Replay for Agent Actions

Priority: Must Understand

Executive Definition: This is the capability to fully reconstruct, after the fact, exactly what an AI agent did, why (what it was told, what it retrieved, what it reasoned), and what real-world effects it had (what systems it touched, what data it read or wrote, what transactions it initiated): for both routine audit and post-incident investigation. It depends directly on the tracing infrastructure above, but adds the requirement that traces be tamper-evident, retained long enough to matter, and usable by security/compliance teams, not just engineers debugging a bug.

Why It Matters: When an agent takes an unauthorized or harmful action (sends an incorrect email, executes an unintended transaction, exposes data it shouldn't have accessed) the organization needs to answer three questions fast: what exactly happened, what was the blast radius, and can it happen again. Unlike a human employee's action, an agent's decision process is not something you can just ask about after the fact; if the trace wasn't captured at execution time, that reasoning is unrecoverable. This makes audit-grade logging a prerequisite for agent authorization scopes in any workflow where the agent can take real-world, irreversible actions.

What I Need to Understand:

• Minimum viable audit trail for an agent action: the triggering user/request, the full prompt context (including any retrieved documents or tool outputs the agent saw), every tool call with arguments and results, the final action taken, and the identity/credentials under which that action executed.
• "Replay" means being able to reconstruct the decision sequence deterministically enough to explain it to an auditor or investigator: this does not require re-running the model (which may give a different output on a re-run given non-determinism), it requires that the original trace is complete enough to stand on its own as the record.
• Retention and tamper-evidence requirements for these logs are a governance/compliance decision (how long, who can access, can they be altered after write) that should be set explicitly, not left as a default from whatever observability tool was adopted for debugging.
• Incident response playbooks for agents need to include agent-specific containment steps (revoking the agent's credentials/tool access, not just "roll back the deployment") because the damage may already be external to the system that was patched.
• This capability should be tested before it's needed: a tabletop exercise where the team actually tries to reconstruct a past agent action from stored traces is the only reliable way to confirm the capability works, rather than assuming it does.

Questions I Should Be Able to Ask My Team:

1. If an agent took a harmful or unauthorized action six months ago, could we reconstruct exactly what it saw, reasoned, and did: today?
2. What is our log retention period for agent traces, and does it meet our actual compliance/audit requirements, or just our debugging convenience window?
3. Does our incident response runbook for agent-caused incidents include revoking the specific agent's tool/credential access, distinct from a general system rollback?

Technologies / Standards / Companies to Know: OpenTelemetry GenAI semantic conventions (as the underlying trace format), NIST AI RMF, immutable/append-only audit logging patterns.

Recommended Learning:

• NIST AI 600-1: Generative AI Profile: governance-level expectations for traceability and incident accountability in generative AI systems.
• Auditing and Logging AI Agent Activity: A Guide for Engineers: LoginRadius Engineering Blog: practitioner-level treatment of what an agent audit trail needs to contain.
• Replay: reconstructing an agent action from the audit log: Deixic: a focused engineering walkthrough of the replay problem specifically.

Time Investment: 2-3 hours