Lab: Prometheus + Thanos Sidecar + MinIO

What you will learn in this TP

• Configure Prometheus for Thanos
• Start the sidecar
• Check the blocks in the bucket

Introduction #

What you will learn in this section

• Configure Prometheus for Thanos
• Start the sidecar
• Check the blocks in the bucket

Context. Prometheus stores its data on local disk for only a few days and does not survive the loss of the server. You add Thanos Sidecar to automatically upload the blocks to shared object storage (MinIO).

Prerequisites: Docker Compose, basic Prometheus knowledge  ·  Estimated time: 50 min

A first end-to-end Thanos pipeline.

Setup #

Compose, configuration and verification.

1. 1. Write the configuration files

   Why? External labels uniquely identify each Prometheus: without them, Thanos can neither tell the sources apart nor deduplicate them.

   Create prometheus.yml (external_labels cluster and replica) and bucket.yml for MinIO.

2. 2. Compose: MinIO, Prometheus and sidecar

   Why? Prometheus must produce uncompacted 2-hour blocks: compaction is delegated to the Thanos Compactor to avoid conflicts.

   Prometheus must produce uncompacted 2-hour blocks; the sidecar shares its volume.

3. 3. Validate

   Why? Checking that the sidecar is ready and can see Prometheus saves you hours hunting for a block that will never be uploaded.

   How do you check that the sidecar is ready and connected to Prometheus?

4. 4. Watch the blocks arrive

   Why? Seeing a block land in the bucket is concrete proof that archiving works; you have to wait for the first block to be closed.

   Once the first block has been closed (2 h), list the bucket.

Difficulty level : ●●○○○ (2/5)