Final Lab: OpenTelemetry Collector to ClickHouse and SQL analyses

What you will learn in this TP

• Configure the ClickHouse exporter
• Write SQL analyses
• Speed up with a materialised view

Introduction #

What you will learn in this section

• Configure the ClickHouse exporter
• Write SQL analyses
• Speed up with a materialised view

Context. Your data team wants to combine telemetry with its own ClickHouse tables. You set up direct OpenTelemetry ingestion and demonstrate the value of SQL.

Prerequisites: Docker Compose  ·  Estimated time: 50 min

Telemetry in ClickHouse, queryable with SQL.

Setup #

Guided steps, with their solutions.

1. 1. Write the Compose file

   Why? ClickHouse and a contrib Collector are all you need.

   Create the Compose file and the Collector configuration.

2. 2. Generate data

   Why? Traces from two services and a few errors give you something to analyse.

   Send traces for two services, some of them with errors.

3. 3. Explore the schema

   Why? Know the columns before writing queries.

   List the tables created and describe otel_traces.

4. 4. Analyse in SQL

   Why? Analytical queries answer operational questions in a single line.

   Calculate, per service, the number of spans, the error rate and the p95 latency.

5. 5. Materialised view and compression

   Why? A pre-aggregated view makes dashboards instant; compression justifies choosing ClickHouse.

   Create a per-minute materialised view, send traces again, query the view, then measure compression.

Difficulty level : ●●●●○ (4/5)