1. The problem to solve #
Linking spans created in different processes.
Why propagate? #
Each service creates its own spans, but to get a single trace you need to pass the trace identity (TraceID) and the parent span on every call. This is called context propagation: the caller injects the context into the message, and the callee extracts it.
2. The W3C Trace Context standard #
traceparent, tracestate and baggage.
The headers #
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
# | | | |
# | trace-id (16 octets, hex) span-id parent flags (01 = échantillonnée)
# version
tracestate: vendor1=valeur,vendor2=autre # informations propres à des fournisseurs
baggage: user.tier=gold,region=eu # paires clé/valeur métier propagées à tous les services| Header | Content | Watch out |
|---|---|---|
traceparent | TraceID, parent SpanID, flags | Essential to link spans |
tracestate | Vendor-specific data | Rarely used in practice |
baggage | Business context (customer, region...) | Visible to all services and third parties: never put secrets in it |
3. Configuring propagators #
Environment variables.
OTEL_PROPAGATORS #
export OTEL_PROPAGATORS=tracecontext,baggage # valeur par défaut : W3C
export OTEL_PROPAGATORS=tracecontext,baggage,b3multi # compatibilité avec des services Zipkin/B3
export OTEL_PROPAGATORS=jaeger,tracecontext # migration depuis JaegerAll the services in the same chain must share at least one common format: it is the first thing to check when a trace is split in two.
# Tester à la main : forcer un TraceID connu et le retrouver dans Tempo
curl -H 'traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' http://localhost:8080/commande
curl -s http://localhost:3200/api/traces/4bf92f3577b34da6a3ce929d0e0e4736 | jq '.batches | length'4. Propagating manually #
Messaging, jobs and uninstrumented HTTP clients.
Injecting and extracting (Python) #
from opentelemetry import trace, propagate, context
tracer = trace.get_tracer("orders")
# Producteur : injecter le contexte dans les en-têtes du message
def publier(message):
with tracer.start_as_current_span("publier_commande", kind=trace.SpanKind.PRODUCER):
headers = {}
propagate.inject(headers) # ajoute traceparent / baggage
kafka_producer.send("commandes", value=message, headers=[(k, v.encode()) for k, v in headers.items()])
# Consommateur : extraire et continuer la trace
def consommer(record):
carrier = {k: v.decode() for k, v in record.headers}
ctx = propagate.extract(carrier)
with tracer.start_as_current_span("traiter_commande", context=ctx, kind=trace.SpanKind.CONSUMER) as span:
span.set_attribute("messaging.destination.name", "commandes")
traiter(record.value)Troubleshooting a broken trace #
| Symptom | Likely cause | Action |
|---|---|---|
| Two traces instead of one | Different propagators between services | Align OTEL_PROPAGATORS |
| Header missing on the callee side | A proxy / gateway strips headers | Allow traceparent and baggage in the proxy configuration |
| Trace lost after a message queue | Context not injected into the message headers | Inject/extract manually or enable the client's instrumentation |
| Orphan spans in background tasks | Context lost between threads/coroutines | Use context.attach() / an instrumented run_in_executor |
| Only some traces are complete | Different head sampling between services | Use parentbased_traceidratio everywhere |
# nginx : laisser passer / générer le contexte
proxy_set_header traceparent $http_traceparent;
proxy_set_header tracestate $http_tracestate;
proxy_set_header baggage $http_baggage;Common mistake: Never put sensitive data in the baggage: it is propagated to all downstream services, including third-party services.
