Lab: Monitor Kubernetes with kube-prometheus-stack

What you will learn in this TP

• Install kube-prometheus-stack
• Declare collection with a ServiceMonitor
• Create a declarative alert

Prometheus in Kubernetes #

What you will learn in this section

• Install kube-prometheus-stack
• Declare collection with a ServiceMonitor
• Create a declarative alert

Context. In Kubernetes pods come and go: editing prometheus.yml by hand no longer makes sense. The Operator discovers targets from Kubernetes objects (ServiceMonitor, PrometheusRule).

Prerequisites : kind, kubectl, helm  ·  Estimated time : 50 min

Operator, ServiceMonitor, alert rule.

1. 1. Install the stack

   Why? The chart installs everything at once (Operator, Prometheus, Alertmanager, Grafana); check pods start before going further.

   Create a kind cluster then install the chart.

2. 2. Deploy an instrumented application

   Why? A named port is what the ServiceMonitor references: without a port name the target is never discovered.

   An application exposing /metrics on a named port.

3. 3. Create the ServiceMonitor

   Why? The ServiceMonitor replaces configuration editing: monitoring becomes a versioned object like the rest.

   Declare collection and check the target in the Prometheus UI.

4. 4. Add an alert rule

   Why? The PrometheusRule makes the alert declarative; the release label must match the Operator's selector, otherwise it is ignored.

   Create a PrometheusRule firing when the error request rate exceeds 5%.

Difficulty level : ●●●○○ (3/5)