RousselTMLEARNING

Article · Cribl: from discovery to expertise

Cribl, the data pipeline

1. Why a data pipeline? #

The problem Cribl solves.

The situation #

Companies send their logs, metrics and traces directly to their tools: SIEM, observability platform, archive. Each source is wired to each tool, each tool bills the volume it receives, and switching tools means reconfiguring hundreds of agents. A data pipeline sits between sources and destinations to route, reduce, enrich and normalize data in one place.

Without and with a pipeline

What you gain #

  • Cost: do not index in the SIEM what is useless there (allowed traffic, empty fields, duplicates); cutting 30 to 50% of the volume sent is common.
  • Freedom: change or add a tool without touching sources; send the same data to two tools during a migration.
  • Quality: normalize formats, add context (team, geolocation, criticality), mask personal data before it leaves the company.
  • Retention: keep 100% of raw data cheaply on object storage and replay it on demand.

2. The Cribl suite #

Five complementary products.

The products #

The Cribl suite
ProductRoleWhere it runs
Cribl Streamthe central pipeline: sources, routes, pipelines, destinations, replayyour servers, Kubernetes or Cribl.Cloud
Cribl Edgelightweight agent: files, journald, Windows events, metrics, Kuberneteson each host or node
Cribl Lakemanaged dataset storage, retention per datasetCribl.Cloud
Cribl Searchfederated search where data lives (S3, Lake, APIs), without re-ingesting itCribl.Cloud
Cribl.Cloudmanaged control plane, workspaces, Copilot (AI assistant)SaaS
Good to know : This course focuses on Cribl Stream, the core of the suite, and covers Edge, Lake and Search in their collection, storage and search roles.

3. The processing model #

Sources, routes, pipelines, destinations.

An event's journey #

Event journey
ConceptDefinition
Sourcedata entry point: listening (syslog, HEC, OTLP) or collecting (S3, REST API)
Routeordered rule: a JavaScript filter, a pipeline, a destination
Pipelinesequence of functions applied to events
Functionunit processing: Parser, Eval, Drop, Mask, Lookup, Aggregations…
Destinationoutput: Splunk, Elastic, Datadog, S3, Kafka, Loki, Cribl Lake…
Packreusable bundle of routes, pipelines and reference data

The Cribl event #

An event is a JSON object. Some fields have special meaning:

FieldMeaning
_rawthe original event text
_timetimestamp in seconds since the Unix epoch
host, source, sourcetype, indexmetadata inherited from the Splunk world, used to filter and route
__inputId, __*internal fields (double underscore): usable in filters, never sent to destinations
cribl_*information added by Cribl (pipeline, breaking)

4. Licensing and versions #

Starting for free.

The offerings #

  • Free: up to 1 TB per day, on Cribl.Cloud or self-hosted. Self-hosted, the free licence requires sending anonymized telemetry to Cribl and is limited to a single Worker Group.
  • Paid (Standard, Enterprise): higher volumes, multiple Worker Groups, RBAC, single sign-on, support; on Cribl.Cloud, usage is counted in credits.
  • Versions: a minor release about every two months (4.17 in March 2026, 4.19 in July, then 4.20); on Cribl.Cloud, a “Regular” or “Slow” release channel per workspace.
Best practice : Cribl University offers free courses and certification tracks (Cribl Certified User, then Admin Stream or Edge, then Engineer): a good complement to this course.

5. Cribl and alternatives #

Where Cribl fits.

Comparison #

ToolStrengthDifference with Cribl Stream
OpenTelemetry Collectoropen standard, free, OTLP-nativeYAML configuration, no UI or data preview, thinner security ecosystem
Bindplanefleet management for OpenTelemetry collectorsbuilt on OpenTelemetry; Cribl has its own engine and event format
Vectorvery fast, open source (Datadog)file-based configuration, no graphical control plane
Logstash, Fluent Bithistorical, free, widespreadno centralized multi-tool routing or built-in replay
Cribl StreamUI with before/after preview, replay, 100+ integrations, SIEM-orientedproprietary software, paid beyond 1 TB per day

Commands to know #

Commands to know
  • docker run -d -p 9000:9000 cribl/cribl:latest : run Cribl Stream locally
  • /opt/cribl/bin/cribl status : service status
  • /opt/cribl/bin/cribl version : installed version
  • sourcetype=='pan:traffic' : example route filter

Sources #

Checked in November 2026 (Cribl 4.20): Cribl ships a minor release about every two months, check the release notes before relying on a detail.

📄 Cribl: the data pipeline for observability and security →

My progress

  • ⏱ Estimated time : ≈ 5 min

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

Cribl Stream and Edge best practices

Best practices for running Cribl in production: route and pipeline design, naming, performance, c...

Archiving, replaying and searching: S3, Replay, Cribl Lake and Cribl Search

Keeping all data cheaply and finding it on demand: partitioned S3 destination, JSON and Parquet f...

Reducing, transforming and optimizing volumes with Cribl

Reducing the volume sent to SIEMs and observability tools without losing useful information: Drop...

Sources, destinations and routes in Cribl Stream

Receiving and sending data with Cribl Stream: push, pull, collector and internal sources, event b...

Cribl Edge: collecting close to the sources

Deploying Cribl Edge on Linux, Windows and Kubernetes: the agent's role, Fleets and Subfleets, co...

Cribl Stream in production: distributed architecture, persistent queues and monitoring

Taking Cribl Stream to production: Leader, Worker Groups and processes, commit and deploy, scalin...

Pipelines and functions: parse, enrich, mask

Building Cribl Stream pipelines: pre-processing, processing and post-processing, the function cat...

Installing and administering Cribl Stream

Installing Cribl Stream with Docker or on Linux, running it as a service, understanding $CRIBL_HO...

Automating Cribl: REST API, Packs, Git and Terraform

Managing Cribl as code: REST API (authentication, group paths, commit and deploy), reusable Packs...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!