Key facts.
- OpenTelemetry publishes GenAI semantic conventions with dedicated agent and framework spans and generative-AI client spans, defining standard attribute names for model, agent, and tool operations.source
- The conventions name tool execution explicitly, with attributes for the tool call (name, call id, arguments, result) so a tool invocation is a first-class span, not an opaque function call.source
- OpenLLMetry, maintained by Traceloop, is an open-source extension that auto-instruments common LLM providers and frameworks and emits standard OpenTelemetry signals plus GenAI attributes.source
- Datadog, New Relic, and Dynatrace support the GenAI semantic conventions natively, so OpenTelemetry-instrumented agent code exports to them without SDK changes (OpenInference, the Arize convention, isn't universally supported).source
Auto-instrumentation or manual: which do you actually need?
Start with auto-instrumentation. OpenLLMetry wraps the LLM client calls, vector database queries, and many agent frameworks for you, emitting spans with token usage and model metadata after a single init call. That covers the model side of the agent with almost no code. OpenInference, the Arize convention, offers AI-tailored span types (LLM, tool, chain, agent, embedding) and suits evaluation-heavy workflows, though its attributes don't always map cleanly onto the OpenTelemetry GenAI conventions, which is why backend support for it's narrower. The honest read: vendor "zero-code" claims come from the library maintainers, but the underlying OpenTelemetry conventions and SDK behavior are standardized and checkable against the official spec, so prototype on your own stack rather than taking a setup demo at face value.
Manual instrumentation covers what auto-instrumentation can't see: your own reasoning steps, your custom tools, and agent state. that's where you open spans by hand and set the convention attributes yourself.
from opentelemetry import trace tracer = trace.get_tracer(__name__)with tracer.start_as_current_span("invoke_agent", attributes={ "gen_ai.operation.name": "invoke_agent", "gen_ai.agent.name": "travel_planner", }) as agent_span:
with tracer.start_as_current_span("chat", attributes={ "gen_ai.operation.name": "chat", "gen_ai.request.model": "the-model-you-call", }) as llm_span: resp = llm_client.chat(...) # the real call llm_span.set_attribute("gen_ai.usage.input_tokens", resp.usage.input_tokens)
with tracer.start_as_current_span("execute_tool get_weather", attributes={ "gen_ai.operation.name": "execute_tool", "gen_ai.tool.name": "get_weather", "gen_ai.tool.call.id": tool_call_id, }) as tool_span: result = weather_api.call(...) # run the tool tool_span.set_attribute("gen_ai.tool.call.result", redact(result))
What do you set on each span, and how do you handle the sensitive parts?
Use the convention attributes so any compliant backend understands your traces. On the agent span, set the operation name and agent name. On each model call, set the operation, the model, and token usage. On each tool, set the tool name, the call id, the arguments, and the result, with redaction for anything sensitive. Use CLIENT span kind for remote calls and INTERNAL for in-process steps. For agent state across steps, carry it as baggage or attach it to a wrapping workflow span. Mask prompts, arguments, and results that contain personal data at capture time, so you aren't paying to store, or exposing, data you never needed in the trace.
| Span | Operation | Attributes that matter |
|---|---|---|
| Agent run | invoke_agent | gen_ai.agent.name, gen_ai.agent.description |
| Model call | chat | gen_ai.request.model, gen_ai.usage.input_tokens / output_tokens |
| Tool call | execute_tool | gen_ai.tool.name, gen_ai.tool.call.id, arguments, result (redacted) |
Instrumenting to a standard is what makes the Pattern Intelligence Layer possible. Reliability at the pattern level needs a consistent, queryable record of how the agent ran (which tools, in what order, with what results), and the GenAI conventions give you exactly that record in a shape that survives a model swap or a backend change. Instrument once against the standard, and the trace you need to spot a recurring failure pattern is already there in a form every tool can read.
Frequently asked questions
Do I have to choose between OpenLLMetry and OpenInference?
For most teams, start with OpenLLMetry because it leans into the OpenTelemetry GenAI conventions and has broad auto-instrumentation. OpenInference suits evaluation-heavy setups but has narrower backend support.
what's the minimum to get value?
Instrument your model calls and one tool, export over OTLP to any backend that supports the GenAI conventions, and look at one full trace. Expand from there.
How do I keep prompts and tool arguments out of my backend?
Redact or mask at capture. Put large or sensitive content in span events you can drop or scrub at the Collector, not in always-indexed attributes.