The correction is integrated into the TP (See the green boxes)
What you will learn in this TP
- Install Cribl Stream with Docker
- Create a syslog source
- Capture events
- Create a destination and a route
- Check in Monitoring
Introduction #
What you will learn in this section
- Install Cribl Stream with Docker
- Create a syslog source
- Capture events
- Create a destination and a route
- Check in Monitoring
Context. Your team wants to evaluate Cribl before trusting it with production logs. You need a test instance and a first end-to-end flow.
Prerequisites: a machine with Docker and Docker Compose (4 GB free memory), curl and jq · Estimated time: 40 min
A persistent Cribl instance receiving syslog and writing it to compressed JSON files.Setup #
Guided steps, with their solution.
1. Start Cribl Stream
Why? The volume keeps configuration if the container is recreated.
Start the container with a configuration volume and an output folder, then wait for the API to answer.See the correction
mkdir -p $HOME/cribl-config $HOME/cribl-out docker run -d --name cribl --restart unless-stopped \ -p 9000:9000 -p 9514:9514 -p 9514:9514/udp -p 8088:8088 \ -e CRIBL_VOLUME_DIR=/opt/cribl/config-volume \ -v $HOME/cribl-config:/opt/cribl/config-volume \ -v $HOME/cribl-out:/data/out \ cribl/cribl:latest until curl -sf http://localhost:9000/api/v1/health >/dev/null; do sleep 2; done; echo ready2. Log in
Why? The default password must be changed immediately.
Openhttp://localhost:9000, log in asadmin/admin, change the password and accept the free licence.See the correction
# Check through the API with the new password export CRIBL_PASSWORD='<new password>' curl -s -X POST localhost:9000/api/v1/auth/login -H 'Content-Type: application/json' \ -d '{"username":"admin","password":"'"$CRIBL_PASSWORD"'"}' | jq -r '.token' | cut -c1-203. Create the syslog source
Why? A source defines the listening port and the fields added to each event.
In Data › Sources › Syslog, createin_syslogon TCP and UDP port 9514, with the fieldsourcetype='syslog'. Send a few messages.See the correction
for i in $(seq 1 50); do logger -n 127.0.0.1 -P 9514 -T -t app "user=u$((i % 5)) action=login status=$([ $((i % 7)) -eq 0 ] && echo failed || echo ok)" done4. Watch live
Why? Always look at data before transforming it.
On the source, open Live Data, resend, then save the capture as the sample filesyslog_app.log. Find_raw,_time,host,appname.See the correction
# Live Data › Start Capture, puis / then / luego: Save as Sample File › syslog_app.log logger -n 127.0.0.1 -P 9514 -T -t app 'user=u1 action=login status=failed'5. Destination and route
Why? The route decides which events go where, with which pipeline.
Create a Filesystem destinationout_files(folder/data/out, JSON, gzip, partitioned bysourcetype), then asyslog_to_filesroute at the top of the table: filter__inputId.startsWith('syslog:in_syslog'), pipelinepassthru, destinationout_files.See the correction
for i in $(seq 1 200); do logger -n 127.0.0.1 -P 9514 -T -t app "user=u$((i % 5)) action=view"; done sleep 70 # the Filesystem destination closes files after its open time find $HOME/cribl-out -name '*.json.gz' | head zcat $(find $HOME/cribl-out -name '*.json.gz' | head -1) | head -2 | jq .6. Check in Monitoring
Why? Counters prove nothing is lost between input and output.
In Monitoring › Data, compare events received byin_syslogand sent byout_files. Then look at the created configuration file.See the correction
find $HOME/cribl-config -name 'routes.yml' -path '*local*' | xargs cat | head -30
Difficulty level : ●●○○○ (2/5)
My progress
Sign in to track your progress and take notes. Sign in
💬 Questions about this lesson
No questions yet: be the first to ask for help or share your configuration.
Recommended Articles
Installing and administering Cribl Stream
Installing Cribl Stream with Docker or on Linux, running it as a service, understanding $CRIBL_HO...
Sources, destinations and routes in Cribl Stream
Receiving and sending data with Cribl Stream: push, pull, collector and internal sources, event b...
Cribl Stream and Edge best practices
Best practices for running Cribl in production: route and pipeline design, naming, performance, c...
Cribl Stream in production: distributed architecture, persistent queues and monitoring
Taking Cribl Stream to production: Leader, Worker Groups and processes, commit and deploy, scalin...
Cribl: the data pipeline for observability and security
Discover Cribl: why a telemetry data pipeline, the Cribl suite (Stream, Edge, Lake, Search, Cribl...
Reducing, transforming and optimizing volumes with Cribl
Reducing the volume sent to SIEMs and observability tools without losing useful information: Drop...
Cribl Edge: collecting close to the sources
Deploying Cribl Edge on Linux, Windows and Kubernetes: the agent's role, Fleets and Subfleets, co...
Pipelines and functions: parse, enrich, mask
Building Cribl Stream pipelines: pre-processing, processing and post-processing, the function cat...
Archiving, replaying and searching: S3, Replay, Cribl Lake and Cribl Search
Keeping all data cheaply and finding it on demand: partitioned S3 destination, JSON and Parquet f...