RousselTMLEARNING

TP · Cribl: from discovery to expertise

Lab: installing Cribl Stream and creating a first flow

What you will learn in this TP
  • Install Cribl Stream with Docker
  • Create a syslog source
  • Capture events
  • Create a destination and a route
  • Check in Monitoring

Introduction #

What you will learn in this section
  • Install Cribl Stream with Docker
  • Create a syslog source
  • Capture events
  • Create a destination and a route
  • Check in Monitoring

Context. Your team wants to evaluate Cribl before trusting it with production logs. You need a test instance and a first end-to-end flow.

Prerequisites: a machine with Docker and Docker Compose (4 GB free memory), curl and jq  ·  Estimated time: 40 min

A persistent Cribl instance receiving syslog and writing it to compressed JSON files.

Setup #

Guided steps, with their solution.

  1. 1. Start Cribl Stream

    Why? The volume keeps configuration if the container is recreated.

    Start the container with a configuration volume and an output folder, then wait for the API to answer.
  2. 2. Log in

    Why? The default password must be changed immediately.

    Open http://localhost:9000, log in as admin / admin, change the password and accept the free licence.
  3. 3. Create the syslog source

    Why? A source defines the listening port and the fields added to each event.

    In Data › Sources › Syslog, create in_syslog on TCP and UDP port 9514, with the field sourcetype = 'syslog'. Send a few messages.
  4. 4. Watch live

    Why? Always look at data before transforming it.

    On the source, open Live Data, resend, then save the capture as the sample file syslog_app.log. Find _raw, _time, host, appname.
  5. 5. Destination and route

    Why? The route decides which events go where, with which pipeline.

    Create a Filesystem destination out_files (folder /data/out, JSON, gzip, partitioned by sourcetype), then a syslog_to_files route at the top of the table: filter __inputId.startsWith('syslog:in_syslog'), pipeline passthru, destination out_files.
  6. 6. Check in Monitoring

    Why? Counters prove nothing is lost between input and output.

    In Monitoring › Data, compare events received by in_syslog and sent by out_files. Then look at the created configuration file.

Difficulty level : ●●○○○ (2/5)

My progress

  • ⏱ Estimated time : ≈ 40 min

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

Installing and administering Cribl Stream

Installing Cribl Stream with Docker or on Linux, running it as a service, understanding $CRIBL_HO...

Sources, destinations and routes in Cribl Stream

Receiving and sending data with Cribl Stream: push, pull, collector and internal sources, event b...

Cribl Stream and Edge best practices

Best practices for running Cribl in production: route and pipeline design, naming, performance, c...

Cribl Stream in production: distributed architecture, persistent queues and monitoring

Taking Cribl Stream to production: Leader, Worker Groups and processes, commit and deploy, scalin...

Cribl: the data pipeline for observability and security

Discover Cribl: why a telemetry data pipeline, the Cribl suite (Stream, Edge, Lake, Search, Cribl...

Reducing, transforming and optimizing volumes with Cribl

Reducing the volume sent to SIEMs and observability tools without losing useful information: Drop...

Cribl Edge: collecting close to the sources

Deploying Cribl Edge on Linux, Windows and Kubernetes: the agent's role, Fleets and Subfleets, co...

Pipelines and functions: parse, enrich, mask

Building Cribl Stream pipelines: pre-processing, processing and post-processing, the function cat...

Archiving, replaying and searching: S3, Replay, Cribl Lake and Cribl Search

Keeping all data cheaply and finding it on demand: partitioned S3 destination, JSON and Parquet f...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!