Set up Agent Analytics

Agent Analytics groups your agent's messages, tool calls, and responses into sessions tied to your users, then scores each session. Setup takes four steps: pick a project, decide your session and privacy strategy, instrument with one path, and verify. For defaults with no customization, complete the Agent Analytics quickstart instead. If you're still evaluating, Analyze agent results shows what Agent Analytics produces.

How setup works

Pick the Amplitude project the data flows into, and get its project API key.
Decide your session ID, user ID, and privacy mode. Refer to Before you instrument.
Instrument your code with one path. Refer to Choose an instrumentation path.
Check your events. Refer to Verify.

Each turn of an instrumented session produces an [Agent] User Message, one [Agent] Tool Call per tool invocation, and an [Agent] AI Response. When the work ends, your code sends [Agent] Session End. After the session closes, Amplitude adds an [Agent] Session Record with the session's enrichment results. The timeline below shows this sequence. Click any event to inspect its shape.

Event type

[Agent] AI Responsefrom the SDK

Fired at

22:33:48

Identity

[Agent] Session ID	4ddcc6b2-1041-432a-aa8c-ebe3eccac40b
[Agent] Agent ID	support-chatbot
[Agent] Trace ID	b4f63d43-d752-4b1f-8489-d234ddf586b2

Event-specific

$llm_message.text	I can help. Your subscription renews on Aug 15…
[Agent] Model Name	gpt-4o-mini
[Agent] Provider	openai
[Agent] Input Tokens	1245
[Agent] Output Tokens	87
[Agent] Latency Ms	3420
[Agent] Cost USD	0.0012

Closes the turn. Carries the eight fields the SDK doctor checks at setup: Session ID, Agent ID, Model, Provider, Latency Ms, Input/Output Tokens, Cost USD. Emitted by s.trackAiMessage(...) or a provider wrapper.

Before you instrument

Decide these before you write code. Each one is hard to change after rollout.

Session ID and user ID: The ID that represents one unit of work, and the stable user ID you already use for product analytics. Refer to Agent Analytics sessions and user identity.
Privacy mode: Whether prompt and response text leaves your infrastructure. Refer to Choose a privacy mode.

Choose a privacy mode

Three modes control what leaves your infrastructure. full (the default) sends message content, metadata_only sends none, and customer_enriched sends none but runs enrichment on labels you provide. Every mode sends tokens, cost, latency, model names, and session grouping. For a full comparison and how to apply each mode on your path, refer to Agent Analytics privacy modes.

Choose an instrumentation path

Pick one path. Every path produces the same [Agent] events, so sessions, enrichment, and charts work the same way.

Path	Use when	Guide
AI SDK (Node, Python)	You can modify a Node or Python codebase	Instrument your agent with the AI SDK
OpenTelemetry	You already export GenAI spans or run an OpenTelemetry Collector	Send OpenTelemetry traces to Agent Analytics
HTTP API or standard Amplitude SDK	The AI SDK doesn't fit your runtime: browsers, edge runtimes, AI app builders, Java, Go, or Ruby	Send agent events without the AI SDK

Connect with the SDK

Install the AI SDK for Node or Python, then let your AI coding agent instrument the app or wire it in by hand. Go to Instrument your agent with the AI SDK.

Send OpenTelemetry traces directly

Point your OTLP exporter or Collector at Amplitude's endpoint and set a conversation attribute on your spans. Go to Send OpenTelemetry traces to Agent Analytics.

Send agent events without the AI SDK

Emit [Agent] events with a standard Amplitude SDK or the HTTP API when the AI SDK doesn't fit your runtime. Go to Send agent events without the AI SDK.

Verify

Send one real interaction and check nine gates on its events in Live Events, then run the production checklist before rollout. Refer to Verify your Agent Analytics instrumentation. If something's missing, refer to Troubleshoot Agent Analytics setup.

Extend your setup

These steps are optional:

Session timeouts: Keep sessions open longer than 30 minutes for jobs that wait on humans. Refer to Configure session timeouts.
User feedback: Send thumbs up, thumbs down, and CSAT ratings as [Agent] Score events. Refer to Collect user feedback.
Session Replay: Link each agent session to the recording of the visit it happened in. Refer to Link agent sessions to Session Replay.
Multi-agent systems: Record delegation from a parent agent to child agents. Refer to Instrument multi-agent systems.
Custom evaluators: Add your own model provider key to run evaluators and calibration runs. Refer to Set up custom evaluators.

Was this helpful?