RousselTMLEARNING

Observability

The error budget: making decisions with SLOs

Calculate and track an error budget, write an error budget policy and use it to arbitrate between new features and reliability.

1. Calculating the budget #

What you are allowed to miss.

Formula #

The error budget is the share of events allowed to fail: 1 - SLO. With a 99.9% SLO over 30 days and 10 million requests, the budget is 10,000 failed requests.

QuantityCalculationExample
Total budget(1 - SLO) × requests0.001 × 10,000,000 = 10,000
Budget consumedobserved errors / total budget4,000 / 10,000 = 40%
Budget remaining1 - consumed60%
Burn rateobserved error rate / (1 - SLO)0.3% / 0.1% = 3
Good to know: A burn rate of 1 consumes exactly the budget over the SLO window. A burn rate of 3 exhausts it in 10 days instead of 30.

2. The error budget policy #

A written agreement between product and operations.

Example policy #

The budget is only valuable if it triggers decisions, decided in advance and signed off by the product and engineering teams:

Budget remainingDecision
> 50%Normal releases, experiments allowed
20 to 50%Normal releases, review of the month's incidents
0 to 20%Priority to reliability work, risky deployments postponed
ExhaustedFeature freeze: only fixes and reliability work are shipped until back within the objective
Good practice: The budget works both ways: a budget that is never consumed means the objective is too cautious or that the team could ship faster.

3. Rolling or calendar windows #

Over what period to measure?

Choosing #

WindowAdvantageDrawback
Rolling (28 or 30 days)Reflects recent experience, no « reset » effectAn incident weighs for 30 days
Calendar (month)Aligned with SLAs and reportsThe budget resets on the 1st, encouraging opportunistic behavior

In practice: a rolling window of 28 or 30 days to steer, a calendar report for the SLA. 28 days contain exactly four weeks, which neutralizes the effect of weekends.

Related articles

Chaos engineering: principles, tools and practice with SLOs

A complete guide to chaos engineering: principles, steady state and hypothesis, blast radius, link with SLOs …

SRE: SLI, SLO and SLA explained simply

The fundamentals of Site Reliability Engineering (SRE): the difference between SLI, SLO and SLA, choosing ind…

Measuring SLIs with Prometheus

Implement availability and latency SLIs with Prometheus: required metrics, ratio queries, histograms, multi-w…

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!