ToolingPublished ExplainedAll news from that day
How an agent trace records each model call and tool call
Chase says LangSmith Engine has scanned over 70 million traces. LangSmith and OpenTelemetry documents show what a trace holds and which fields are opt-in.
Check our sources · 11 facts from 4 sources
Illustration made with AI for ai notis
Key points
- LangSmith defines a trace as a collection of runs for a single operation; a run is one unit of work, such as calling an LLM or retrieving documents.
- OpenTelemetry says a span, its unit of work, includes a name, parent span ID, start and end timestamps, attributes, events and a status of Unset, Error or Ok.
- In OpenTelemetry's GenAI conventions, tool call arguments and results are opt-in attributes, and the arguments may contain sensitive information.
What happened
LangChain's Harrison Chase told the Interrupt conference in London that its Engine tool had scanned over 70 million traces and detected over 21,000 issues, which are his figures about his own product. A trace is the record such tools read.
LangSmith's documentation defines a trace as a collection of runs for a single operation, and a run as a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents.
Its own example is a user request that triggers an agent that calls a model, runs a tool, and then calls the model again: all of those runs belong to the same trace. LangSmith says a run can be thought of as an OpenTelemetry span, and that each trace is limited to a maximum of 25,000 runs.
OpenTelemetry says a span represents a unit of work or operation, and that a span includes a name, a parent span ID (empty for root spans), start and end timestamps, attributes, span events and a status. The status is Unset, Error or Ok.
For agents, OpenTelemetry's GenAI conventions define spans named invoke_agent for an agent call and execute_tool for a tool call, with the tool's name as a required attribute and input and output token counts as recommended ones.
The conventions make tool call arguments and results opt-in, and warn that arguments may contain sensitive information. The agent-span conventions carry the status Development.
LangSmith says its integrations for frameworks such as LangChain, LangGraph, OpenAI, Anthropic and CrewAI capture inputs, outputs and metadata without manual code changes.
Underlined sentences link to the facts we checked. Dotted words open a short explainer.
Fact 2 of 11 · from LangChain
A run represents a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents. ... A trace is a collection of runs for a single operation.
Fact 3 of 11 · from LangChain
For example, if a user request triggers an agent that calls a model, runs a tool, and then calls the model again, all of those runs belong to the same trace.
Fact 4 of 11 · from LangChain
If you are familiar with OpenTelemetry, you can think of a run as a span. ... Each trace is limited to a maximum of 25,000 runs.
Fact 5 of 11 · from LangChain
When you use a supported framework such as LangChain, LangGraph, OpenAI, Anthropic, or CrewAI, the integration captures inputs, outputs, and metadata without requiring manual code changes.
What it means for you
Our viewA trace is only as useful as the fields it holds, and the conventions leave some of the most useful ones optional. Because tool call arguments and results are opt-in, a trace can show that a tool ran without showing what it was given.
LangSmith says its integrations capture inputs, outputs and metadata without manual code changes, so the default may collect more than you expect. Before you switch tracing on for real work, list which fields leave your systems and who can read them.
It adds no new facts.
Your reaction
We count reactions per story and day, never who reacted. The counts help us choose what goes in the monthly issue. If you are signed in, your own page shows yours too.
We count reactions per story and day, never who reacted. They help choose what goes in the monthly issue.
Check our sources
Every sentence above is checked against these 4 sources.
1 Why Own Your Intelligence? Harrison Chase Keynote | Interrupt London
Open the source archived copy-
At 31:50 Harrison Chase says of LangSmith Engine: We've seen it be used tremendously already so far, with over 70 million traces scanned and detected over 21,000 issues. The figures are his, about his own product.
cite
2 LangSmith observability concepts
Open the source archived copy-
LangSmith's observability concepts page defines a trace as a collection of runs for a single operation, and a run as a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents.
citeA run represents a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents. ... A trace is a collection of runs for a single operation.
-
The page gives as an example a user request that triggers an agent that calls a model, runs a tool, and then calls the model again, and says all of those runs belong to the same trace.
citeFor example, if a user request triggers an agent that calls a model, runs a tool, and then calls the model again, all of those runs belong to the same trace.
-
The page says that for readers familiar with OpenTelemetry, a run can be thought of as a span, and that each trace is limited to a maximum of 25,000 runs.
citeIf you are familiar with OpenTelemetry, you can think of a run as a span. ... Each trace is limited to a maximum of 25,000 runs.
-
The page says that with integrations for frameworks such as LangChain, LangGraph, OpenAI, Anthropic or CrewAI, the integration captures inputs, outputs, and metadata without requiring manual code changes.
citeWhen you use a supported framework such as LangChain, LangGraph, OpenAI, Anthropic, or CrewAI, the integration captures inputs, outputs, and metadata without requiring manual code changes.
3 Traces (OpenTelemetry concepts)
Open the source archived copy-
OpenTelemetry's traces concept page says a span represents a unit of work or operation, and lists what a span includes: Name, Parent span ID (empty for root spans), Start and End Timestamps, Span Context, Attributes, Span Events, Span Links and Span Status.
citeA span represents a unit of work or operation. Spans are the building blocks of Traces. In OpenTelemetry, they include the following information: Name Parent span ID (empty for root spans) Start and End Timestamps Span Context Attributes Span Events Span Links Span Status
-
The page says the three possible values of a span status are Unset, Error and Ok.
citeEach span has a status. The three possible values are: Unset Error Ok The default value is Unset.
4 Semantic Conventions for GenAI agent and framework spans
Open the source archived copy-
OpenTelemetry's GenAI agent spans conventions set the operation name invoke_agent for an agent invocation, point tool calls to the Execute Tool Span, and in their tool-call refinements give the span name execute_tool followed by the tool name and list gen_ai.tool.name as a Required attribute.
Toned down to what the source says
citeThe `gen_ai.operation.name` SHOULD be `invoke_agent`. ... If you are using tools in your agent, refer to Execute Tool Span (/docs/gen-ai/gen-ai-spans.md#execute-tool-span) and the refinements below. ... **Span name** SHOULD be `execute_tool {gen_ai.tool.name}`. ... `gen_ai.tool.name` | Development | `Required` | string | Name of the tool utilized by the agent.
-
The same conventions list gen_ai.usage.input_tokens and gen_ai.usage.output_tokens as Recommended attributes of the agent invocation span.
cite`gen_ai.usage.input_tokens` | Development | `Recommended` | int | The total number of input (prompt) tokens across all modalities. ... `gen_ai.usage.output_tokens` | Development | `Recommended` | int | The total number of output (completion) tokens across all modalities.
-
The conventions list gen_ai.tool.call.arguments and gen_ai.tool.call.result as Opt-In attributes, and warn that the arguments attribute may contain sensitive information.
cite`gen_ai.tool.call.arguments` | Development | `Opt-In` | any | Parameters passed to the tool call. ... `gen_ai.tool.call.result` | Development | `Opt-In` | any | The result returned by the tool call ... [10] `gen_ai.tool.call.arguments`: > [!WARNING] > This attribute may contain sensitive information.
-
The document gives its status as Development.
cite# Semantic Conventions for GenAI agent and framework spans **Status**: [Development][DocumentStatus]
Every sentence above is checked against these 4 sources. Each fact has its own address you can cite.
Source 1 · LangChain (YouTube) · 30 Oct 2026
Why Own Your Intelligence? Harrison Chase Keynote | Interrupt London
At 31:50 Harrison Chase says of LangSmith Engine: We've seen it be used tremendously already so far, with over 70 million traces scanned and detected over 21,000 issues. The figures are his, about his own product.
Source 2 · LangChain
LangSmith observability concepts
LangSmith's observability concepts page defines a trace as a collection of runs for a single operation, and a run as a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents.
A run represents a single unit of work executed by an agent, such as calling an LLM, formatting a prompt, or retrieving documents. ... A trace is a collection of runs for a single operation.
The page gives as an example a user request that triggers an agent that calls a model, runs a tool, and then calls the model again, and says all of those runs belong to the same trace.
For example, if a user request triggers an agent that calls a model, runs a tool, and then calls the model again, all of those runs belong to the same trace.
The page says that for readers familiar with OpenTelemetry, a run can be thought of as a span, and that each trace is limited to a maximum of 25,000 runs.
If you are familiar with OpenTelemetry, you can think of a run as a span. ... Each trace is limited to a maximum of 25,000 runs.
Show all 4 factsShow fewer facts
The page says that with integrations for frameworks such as LangChain, LangGraph, OpenAI, Anthropic or CrewAI, the integration captures inputs, outputs, and metadata without requiring manual code changes.
When you use a supported framework such as LangChain, LangGraph, OpenAI, Anthropic, or CrewAI, the integration captures inputs, outputs, and metadata without requiring manual code changes.
Source 3 · OpenTelemetry
Traces (OpenTelemetry concepts)
OpenTelemetry's traces concept page says a span represents a unit of work or operation, and lists what a span includes: Name, Parent span ID (empty for root spans), Start and End Timestamps, Span Context, Attributes, Span Events, Span Links and Span Status.
A span represents a unit of work or operation. Spans are the building blocks of Traces. In OpenTelemetry, they include the following information: Name Parent span ID (empty for root spans) Start and End Timestamps Span Context Attributes Span Events Span Links Span Status
The page says the three possible values of a span status are Unset, Error and Ok.
Each span has a status. The three possible values are: Unset Error Ok The default value is Unset.
Source 4 · OpenTelemetry (GitHub)
Semantic Conventions for GenAI agent and framework spans
OpenTelemetry's GenAI agent spans conventions set the operation name invoke_agent for an agent invocation, point tool calls to the Execute Tool Span, and in their tool-call refinements give the span name execute_tool followed by the tool name and list gen_ai.tool.name as a Required attribute.
Toned down to what the source saysThe `gen_ai.operation.name` SHOULD be `invoke_agent`. ... If you are using tools in your agent, refer to Execute Tool Span (/docs/gen-ai/gen-ai-spans.md#execute-tool-span) and the refinements below. ... **Span name** SHOULD be `execute_tool {gen_ai.tool.name}`. ... `gen_ai.tool.name` | Development | `Required` | string | Name of the tool utilized by the agent.
The same conventions list gen_ai.usage.input_tokens and gen_ai.usage.output_tokens as Recommended attributes of the agent invocation span.
`gen_ai.usage.input_tokens` | Development | `Recommended` | int | The total number of input (prompt) tokens across all modalities. ... `gen_ai.usage.output_tokens` | Development | `Recommended` | int | The total number of output (completion) tokens across all modalities.
The conventions list gen_ai.tool.call.arguments and gen_ai.tool.call.result as Opt-In attributes, and warn that the arguments attribute may contain sensitive information.
`gen_ai.tool.call.arguments` | Development | `Opt-In` | any | Parameters passed to the tool call. ... `gen_ai.tool.call.result` | Development | `Opt-In` | any | The result returned by the tool call ... [10] `gen_ai.tool.call.arguments`: > [!WARNING] > This attribute may contain sensitive information.
Show all 4 factsShow fewer facts
The document gives its status as Development.
# Semantic Conventions for GenAI agent and framework spans **Status**: [Development][DocumentStatus]
Topics
The morning email
On the mornings we publish: the three top stories and up to four short ones. Free.
We email you a link to confirm. An issue may include one sponsor, always labelled Sponsored · Advertisement. Our emails count opens and clicks, not who made them. Unsubscribe in one click. What we keep