End-to-end CI/CD (4/5): deployment strategies and rollback

1. The strategies #

Choosing based on risk and infrastructure.

Overview #

Rolling update #

# Kubernetes : jamais moins de 100 % de capacité pendant la mise à jour
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate: { maxSurge: 1, maxUnavailable: 0 }
  template:
    spec:
      containers:
        - name: api
          image: registry.example.com/shop/api:3f9c2e1
          readinessProbe:                     # le trafic n'arrive qu'aux pods prêts
            httpGet: { path: /health, port: 8080 }
            periodSeconds: 5

Good to know: Without a readiness probe, Kubernetes sends traffic to a pod that is still starting: the rolling update then produces errors.

Blue/green #

Canary #

# Argo Rollouts : canary avec paliers et analyse automatique
strategy:
  canary:
    steps:
      - setWeight: 5
      - pause: { duration: 5m }
      - analysis: { templates: [{ templateName: taux-erreur }] }
      - setWeight: 20
      - pause: { duration: 10m }
      - setWeight: 50
      - pause: { duration: 15m }

2. Decoupling deployment from release #

Feature flags and dark launch.

Feature flags #

A feature flag lets you deploy inactive code and enable it later, for a percentage of users, a region or internal testers only. It is what makes trunk-based development with unfinished features possible.

if flags.is_enabled("nouveau-panier", user_id=user.id):   # OpenFeature, Unleash, LaunchDarkly, Flagsmith...
    return nouveau_panier(user)
return ancien_panier(user)

Common mistake: Every flag is debt: remove it as soon as the feature is enabled for everyone, otherwise the code accumulates dead branches that are hard to test.

Dark launch #

The new version receives a copy of real traffic (mirroring) without its responses being returned to users: performance and errors are measured without risk.

3. Verifying and rolling back #

A deployment is only finished once verified.

Automatic checks #

1. Smoke tests right after deployment: the home page responds, login works, the health API is green.
2. Metric analysis during ramp-up: error rate, p95 latency, saturation, compared with the previous version.
3. Automatic decision: continue, pause or abort.

# smoke test minimal dans le pipeline
for i in $(seq 1 10); do
  curl -fsS --max-time 3 https://staging.example.com/health && exit 0
  sleep 6
done
echo "smoke test en échec" >&2 ; exit 1

Rollback or roll forward #

kubectl rollout undo deployment/api
kubectl rollout history deployment/api
helm rollback shop 41
git revert <sha> && git push      # GitOps : l'annulation passe aussi par Git