RousselTMLEARNING
EN
FrançaisEnglishEspañol
Sign in

Observabilité · Courses

Alertmanager and alert management: from discovery to expertise

A complete course on alerting: Prometheus rules, Alertmanager (routing, grouping, silences, inhibitions, high availability), Grafana Alerting and on-call organisation.

8 chapters13 lessonsEN
At your pace, through practice.

By the end of this course you will be able to

  • Write and test relevant alerting rules
  • Configure Alertmanager routing, grouping and receivers
  • Reduce noise with silences, inhibitions and time intervals
  • Make alerting highly available and monitored
  • Provision Grafana Alerting and organise on-call

🎯 Level : Intermediate👥 Who it is for : SREs, DevOps engineers and operators who want useful, well-routed and reliable alerts

LEARNING PATH

Course curriculum

0Course overview1 lesson
  1. Assessment quizQuiz · ⏱ 10 min
1Introduction1 lesson

The role of Alertmanager and the lifecycle of an alert.

  1. Alertmanager: role and lifecycleArticle · ⏱ 6 min
2Alerting rules2 lessons

Designing and testing rules.

  1. Lab: First Prometheus, Alertmanager and email alerting pipelineHands-on lab · ⏱ 40 min · Level 2/5
  2. Writing good alerting rulesArticle · ⏱ 6 min
3Routing and receivers3 lessons

Routing tree, grouping and templates.

  1. Routing, grouping and receiversArticle · ⏱ 9 min
  2. Lab: Routing by team, grouping and inhibitionHands-on lab · ⏱ 40 min · Level 3/5
  3. Lab: Rule unit tests and maintenance silencesHands-on lab · ⏱ 35 min · Level 3/5
4Reducing noise2 lessons

Silences, inhibitions and time intervals.

  1. Silences, inhibitions and intervalsArticle · ⏱ 7 min
  2. Lab: Highly available Alertmanager and WatchdogHands-on lab · ⏱ 45 min · Level 4/5
5High availability and operations1 lesson

Cluster, Watchdog and troubleshooting.

  1. High availability and operationsArticle · ⏱ 4 min
6Grafana Alerting and on-call2 lessons

Multi-source alerting and human organisation.

  1. Grafana Alerting and on-callArticle · ⏱ 4 min
  2. Final Lab: Grafana alerting provisioned in YAMLHands-on lab · ⏱ 45 min · Level 3/5
7Best practices1 lesson

Recommended practices for running the product in production.

  1. Alerting best practicesArticle · ⏱ 4 min

LEARN BY DOING

Use cases & applications

Routing by team

Each team receives its alerts on its own channel.

Noise-free maintenance

Targeted silences and time intervals.

Cascading failures

A primary alert silences the secondary alerts.

Reliable on-call

Highly available alerting and Watchdog.

Recommended prerequisites

Prometheus: From Discovery to Expertise

What next?

SRE in practice: SLIs, SLOs and error budgetsGrafana Mimir: from discovery to expertiseGrafana: From Discovery to Expertise

Frequently asked questions

Does Alertmanager evaluate rules?

No: Prometheus (or Mimir, Loki, Grafana) evaluates the rules; Alertmanager receives the alerts and handles grouping, routing and notifications.

How do I avoid duplicate alerts?

Alertmanager deduplicates identical alerts; in a cluster, the members coordinate so that only one notification is sent.

Grafana Alerting or Alertmanager?

Alertmanager for Prometheus rules managed as code; Grafana Alerting for multi-source rules. Keep a single place for your routes.

How do I reduce alert fatigue?

Alert on symptoms, group, inhibit, mute warnings at night and review non-actionable alerts every week.