OpenInference: the OpenTelemetry conventions for tracing AI applications

1. What is OpenInference? #

A common vocabulary for AI traces.

Definition #

OpenInference is an open source project (initiated by Arize) that defines two things:

• semantic conventions: which attributes to put on a span to describe a model call, a document retrieval, a tool call or an agent step;
• ready-to-use instrumentations that automatically produce these spans for the main AI SDKs and frameworks, in Python, JavaScript and Java.

Everything is built on OpenTelemetry: OpenInference spans are ordinary OpenTelemetry spans, exported over OTLP. Phoenix and Arize AX can display them richly (messages, documents, tokens), and any OTLP backend can store them.

Why conventions? #

Without conventions, each team names its attributes its own way (prompt, llm_input, messages...) and no tool can compute a token cost or display a conversation. With OpenInference, llm.token_count.prompt means the same thing whatever the framework, which enables generic dashboards, evaluations and comparisons.

2. Span kinds #

openinference.span.kind.

The values #

Good to know: The list of kinds evolves with the specification (for example DECISION for routing choices): see spec/semantic_conventions.md in the OpenInference repository.

3. Essential attributes #

What a span contains.

By family #

4. Instrumenting #

Automatic first, then manual.

Automatic instrumentation #

pip install arize-phoenix-otel openinference-instrumentation-openai openai

from phoenix.otel import register

# configure OpenTelemetry et active toutes les instrumentations OpenInference installées
tracer_provider = register(
    project_name="support-bot",
    endpoint="http://localhost:6006/v1/traces",   # Phoenix, en OTLP/HTTP
    auto_instrument=True,
)

from openai import OpenAI
client = OpenAI()
client.chat.completions.create(model="gpt-4o-mini",
                               messages=[{"role": "user", "content": "Bonjour"}])

Without Phoenix, the same instrumentation works with the standard OpenTelemetry SDK:

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(endpoint="http://collector:4318/v1/traces")))
OpenAIInstrumentor().instrument(tracer_provider=provider)

Manual instrumentation #

For an in-house step (a search in your own index, a business tool), create the span yourself following the conventions:

from opentelemetry import trace
from openinference.semconv.trace import SpanAttributes, OpenInferenceSpanKindValues

tracer = trace.get_tracer("rag")

def retrieve(question: str):
    with tracer.start_as_current_span("retrieve") as span:
        span.set_attribute(SpanAttributes.OPENINFERENCE_SPAN_KIND, OpenInferenceSpanKindValues.RETRIEVER.value)
        span.set_attribute(SpanAttributes.INPUT_VALUE, question)
        docs = index.search(question, k=3)
        for i, d in enumerate(docs):
            span.set_attribute(f"retrieval.documents.{i}.document.id", d.id)
            span.set_attribute(f"retrieval.documents.{i}.document.content", d.text)
            span.set_attribute(f"retrieval.documents.{i}.document.score", d.score)
        return docs

Context: sessions, users, metadata #

from openinference.instrumentation import using_attributes

with using_attributes(session_id=conversation_id, user_id=user.id,
                      metadata={"app_version": "2.3.0", "feature": "faq"}, tags=["beta"]):
    answer = agent.run(question)   # tous les spans créés ici portent ces attributs

Good practice: Always set session.id for conversations and a version metadata: this is what makes it possible to display a complete conversation and compare two versions of the application.

5. Privacy and volume #

Prompts contain sensitive data.

Masking #

Prompts and answers may contain personal data. OpenInference instrumentations accept a masking configuration, in code (TraceConfig) or through environment variables:

Common mistake: Hiding all inputs then makes it impossible to evaluate answers. Prefer masking sensitive fields (in code or in an OpenTelemetry Collector) and restricting access to traces.

6. OpenInference and OpenTelemetry's GenAI conventions #

Two close standards.

Comparison #

OpenTelemetry also defines conventions for generative AI (gen_ai.* attributes: gen_ai.request.model, gen_ai.usage.input_tokens...), still being stabilized. OpenInference came earlier, is richer for RAG and agents (documents, reranking, tools, span kinds), and is widely tooled.

Good practice: Choose the instrumentation based on your main backend; if you need to feed several tools, an OpenTelemetry Collector can rename attributes (transform processor) to switch from one convention to the other.