What you will learn in this TP
- Deploy Leader and Workers
- Configure a group and deploy
- Enable a persistent queue
- Simulate a destination outage
- Automate commit and deploy through the API
Introduction #
What you will learn in this section
- Deploy Leader and Workers
- Configure a group and deploy
- Enable a persistent queue
- Simulate a destination outage
- Automate commit and deploy through the API
Context. Before production, the team demands two proofs: configuration deploys in a controlled way to every Worker, and a SIEM outage loses no event.
Prerequisites: a machine with Docker and Docker Compose (4 GB free memory), curl and jq · Estimated time: 75 min
A three-node Cribl cluster, a version deployed through the UI then the API, and a persistent queue drained without loss after an outage.Setup #
Guided steps, with their solution.
1. Start the cluster
Why? The shared token authenticates Workers to the Leader.
Write the Compose file (Leader, two Workers, fake SIEM), start it and check in Manage › Workers that both Workers are connected to thedefaultgroup.2. Configure and deploy
Why? Nothing changes on Workers until the version is deployed.
In thedefaultgroup: Datagen sourcein_gen(a bundled sample, 50 events per second), Webhook destinationout_siemtohttp://siem:8080/, routegen_siem. Click Commit then Deploy.3. Enable the persistent queue
Why? Without a queue, a destination outage blocks or drops data.
Onout_siem, set Backpressure behavior to Persistent Queue (max size 1 GB, gzip compression), then Commit and Deploy.4. Simulate the outage
Why? Only a real outage proves the behaviour.
Stop the SIEM for 2 minutes, watch the queue grow on the Workers, restart it and check it drains.5. Deploy through the API
Why? In production, changes go through CI, not clicks.
Change thein_genrate through the API, then chain group commit, deploy and Leader commit. Check the deployed version.
Difficulty level : ●●●●○ (4/5)
My progress
Sign in to track your progress and take notes. Sign in
💬 Questions about this lesson
No questions yet: be the first to ask for help or share your configuration.
Recommended Articles
Installing and administering Cribl Stream
Installing Cribl Stream with Docker or on Linux, running it as a service, understanding $CRIBL_HO...
Distributed tracing: context propagation (W3C Trace Context and baggage)
Understand and implement OpenTelemetry context propagation: the traceparent header, tracestate, b...
Cribl: the data pipeline for observability and security
Discover Cribl: why a telemetry data pipeline, the Cribl suite (Stream, Edge, Lake, Search, Cribl...
Sources, destinations and routes in Cribl Stream
Receiving and sending data with Cribl Stream: push, pull, collector and internal sources, event b...
Pipelines and functions: parse, enrich, mask
Building Cribl Stream pipelines: pre-processing, processing and post-processing, the function cat...
Reducing, transforming and optimizing volumes with Cribl
Reducing the volume sent to SIEMs and observability tools without losing useful information: Drop...
Distributed tracing best practices with Jaeger
Checklist for useful tracing: OpenTelemetry instrumentation, span naming, attributes, propagation...
Datadog on Kubernetes: Operator, Cluster Agent and Autodiscovery
Monitor Kubernetes with Datadog: the Datadog Operator and the DatadogAgent resource, Cluster Agen...
Docker best practices: images, runtime, security and operations
Docker best practices: Dockerfile and multi-stage builds, minimal and reproducible images, unpriv...