RousselTMLEARNING

TP · OpenTelemetry: From Discovery to Expertise

Lab: Hand-instrument traces (spans, events, errors, links, sampling)

What you will learn in this TP
  • Create spans with the SDK
  • Report errors, events and links
  • Query with TraceQL and sample

Introduction #

What you will learn in this section
  • Create spans with the SDK
  • Report errors, events and links
  • Query with TraceQL and sample

Context. The order service has no auto-instrumented library for its business logic. You instrument it by hand to see an order's steps, its payment errors and its batch processing.

Prerequisites: Docker and Python 3.10 or later; the grafana/otel-lgtm stack (Collector, Tempo, Loki, Prometheus, Pyroscope and Grafana in one container)  ·  Estimated time: 50 min

Rich, queryable traces and controlled sampling.

Hands-on #

Guided steps, with their solution.

  1. 1. Start the observability stack

    Why? A single image provides an OTLP Collector and every backend: focus on the signal.

    Start grafana/otel-lgtm, create a Python environment and point exporters to OTLP/HTTP port 4318.
  2. 2. Initialise the SDK

    Why? The TracerProvider holds the resource and exporter; the BatchSpanProcessor sends in batches.

    Install the SDK and the OTLP/HTTP exporter, then write otel_setup.py configuring the resource (service.name, service.version, deployment.environment.name) and export.
  3. 3. Create a business trace

    Why? Spans named per step, with SpanKind and convention-compliant attributes, make the trace readable by every tool.

    Write commande.py: a root span traiter commande and three children (stock as CLIENT with database attributes, computation, payment), an event, and a payment error one time in three with record_exception and ERROR status.
  4. 4. Explore and query with TraceQL

    Why? TraceQL filters on span structure and attributes: one query finds the traces that matter.

    In Grafana, open Explore > Tempo. Run the queries below, open a failed trace and find the event and exception of the payer span.
  5. 5. Link a batch job with links

    Why? A batch processing items from several traces cannot have several parents: use links.

    Write lot.py: three independent "reception" traces, then a facturer lot span linked to all three. Find the links in the span details.
  6. 6. Sample

    Why? Ratio sampling reduces volume; parentbased ensures a trace is kept or dropped as a whole.

    Re-run commande.py with 20% sampling and compare the number of traces received; check no trace is partial. Explain why Collector tail sampling would be better to keep all errors.

Difficulty level : ●●●○○ (3/5)

My progress

  • ⏱ Estimated time : ≈ 50 min

Sign in to track your progress and take notes. Sign in

💬 Recent discussions about this course

Ask a question about this lesson →See all discussions about this course

Recommended Articles

OpenTelemetry traces in depth: spans, status, events, links and sampling

Master the trace signal: anatomy of a distributed trace, span and SpanKind, attributes, status an...

Distributed tracing: context propagation (W3C Trace Context and baggage)

Understand and implement OpenTelemetry context propagation: the traceparent header, tracestate, b...

TraceQL: the trace query language

Master TraceQL: span selectors, resource and span attributes, operators, duration comparisons, st...

OpenTelemetry: automatic vs manual instrumentation

Understand the differences between automatic and manual instrumentation with OpenTelemetry, their...

Grafana Alloy: Trace Management with Tempo

Dive into distributed trace processing. Learn how to ingest OTLP, Jaeger, or Zipkin traces with G...

Logs with OpenTelemetry: data model, appenders, correlation and collection

Master the log signal: the LogRecord data model, the Logs Bridge API and appenders (Logback, Log4...

Instrumenting applications for Jaeger with OpenTelemetry

Send traces to Jaeger with OpenTelemetry: auto-instrumentation (Java, Python, Node.js), SDK and m...

Grafana Tempo best practices: instrumentation, sampling, queries and costs

Tempo best practices: attribute conventions, useful instrumentation, sampling, using TraceQL, cor...

OpenTelemetry semantic conventions (semconv): naming telemetry

Understand and apply OpenTelemetry semantic conventions: resource, HTTP, database, messaging and ...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!