RousselTMLEARNING

Article · Cribl: from discovery to expertise

Cribl, the data pipeline

1. Why a data pipeline? #

The problem Cribl solves.

The situation #

Companies send their logs, metrics and traces directly to their tools: SIEM, observability platform, archive. Each source is wired to each tool, each tool bills the volume it receives, and switching tools means reconfiguring hundreds of agents. A data pipeline sits between sources and destinations to route, reduce, enrich and normalize data in one place.

Without and with a pipeline

What you gain #

  • Cost: do not index in the SIEM what is useless there (allowed traffic, empty fields, duplicates); cutting 30 to 50% of the volume sent is common.
  • Freedom: change or add a tool without touching sources; send the same data to two tools during a migration.
  • Quality: normalize formats, add context (team, geolocation, criticality), mask personal data before it leaves the company.
  • Retention: keep 100% of raw data cheaply on object storage and replay it on demand.

2. The Cribl suite #

Five complementary products.

The products #

The Cribl suite
ProductRoleWhere it runs
Cribl Streamthe central pipeline: sources, routes, pipelines, destinations, replayyour servers, Kubernetes or Cribl.Cloud
Cribl Edgelightweight agent: files, journald, Windows events, metrics, Kuberneteson each host or node
Cribl Lakemanaged dataset storage, retention per datasetCribl.Cloud
Cribl Searchfederated search where data lives (S3, Lake, APIs), without re-ingesting itCribl.Cloud
Cribl.Cloudmanaged control plane, workspaces, Copilot (AI assistant)SaaS
Good to know : This course focuses on Cribl Stream, the core of the suite, and covers Edge, Lake and Search in their collection, storage and search roles.

3. The processing model #

Sources, routes, pipelines, destinations.

An event's journey #

Event journey
ConceptDefinition
Sourcedata entry point: listening (syslog, HEC, OTLP) or collecting (S3, REST API)
Routeordered rule: a JavaScript filter, a pipeline, a destination
Pipelinesequence of functions applied to events
Functionunit processing: Parser, Eval, Drop, Mask, Lookup, Aggregations…
Destinationoutput: Splunk, Elastic, Datadog, S3, Kafka, Loki, Cribl Lake…
Packreusable bundle of routes, pipelines and reference data

The Cribl event #

An event is a JSON object. Some fields have special meaning:

FieldMeaning
_rawthe original event text
_timetimestamp in seconds since the Unix epoch
host, source, sourcetype, indexmetadata inherited from the Splunk world, used to filter and route
__inputId, __*internal fields (double underscore): usable in filters, never sent to destinations
cribl_*information added by Cribl (pipeline, breaking)

4. Licensing and versions #

Starting for free.

The offerings #

  • Free: up to 1 TB per day, on Cribl.Cloud or self-hosted. Self-hosted, the free licence requires sending anonymized telemetry to Cribl and is limited to a single Worker Group.
  • Paid (Standard, Enterprise): higher volumes, multiple Worker Groups, RBAC, single sign-on, support; on Cribl.Cloud, usage is counted in credits.
  • Versions: a minor release about every two months (4.17 in March 2026, 4.19 in July, then 4.20); on Cribl.Cloud, a “Regular” or “Slow” release channel per workspace.
Best practice : Cribl University offers free courses and certification tracks (Cribl Certified User, then Admin Stream or Edge, then Engineer): a good complement to this course.

5. Cribl and alternatives #

Where Cribl fits.

Comparison #

ToolStrengthDifference with Cribl Stream
OpenTelemetry Collectoropen standard, free, OTLP-nativeYAML configuration, no UI or data preview, thinner security ecosystem
Bindplanefleet management for OpenTelemetry collectorsbuilt on OpenTelemetry; Cribl has its own engine and event format
Vectorvery fast, open source (Datadog)file-based configuration, no graphical control plane
Logstash, Fluent Bithistorical, free, widespreadno centralized multi-tool routing or built-in replay
Cribl StreamUI with before/after preview, replay, 100+ integrations, SIEM-orientedproprietary software, paid beyond 1 TB per day

Commands to know #

Commands to know
  • docker run -d -p 9000:9000 cribl/cribl:latest : run Cribl Stream locally
  • /opt/cribl/bin/cribl status : service status
  • /opt/cribl/bin/cribl version : installed version
  • sourcetype=='pan:traffic' : example route filter

Sources #

Checked in November 2026 (Cribl 4.20): Cribl ships a minor release about every two months, check the release notes before relying on a detail.

📄 Cribl: the data pipeline for observability and security →

My progress

  • ⏱ Estimated time : ≈ 5 min

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

Bindplane best practices

Checklist for operating Bindplane: organizing by labels and fleets, measured reduction, masking a...

Grafana Alloy: Log Management with Loki

Discover how to configure Grafana Alloy to read log files, journald, or network streams, process ...

Security with Datadog: Cloud SIEM, Cloud Security and application protection

Discover Datadog's security products: Cloud SIEM and detection rules, Cloud Security (posture, vu...

Agent policies, integrations, data streams and pipelines

Build agent policies, add integrations (System, Nginx, Custom Logs), choose namespaces, customise...

Vector: transforms in detail (remap and VRL, route, filter, reduction and metrics)

Transform events in Vector: remap and the VRL language (parsing, enrichment, error handling), rou...

OpenTelemetry Collector, Grafana Alloy and Bindplane: comparing three approaches to telemetry pipelines

Detailed comparison of three ways to manage telemetry pipelines: the OpenTelemetry Collector in Y...

Dynatrace security and governance: advanced IAM, policies, SSO, audit and data protection

Secure and govern Dynatrace: SSO and SCIM, groups, IAM policies with conditions, security context...

Reducing, masking and routing telemetry with Bindplane

Reduce ingestion costs with Bindplane: severity filters, deduplication, sampling, field removal, ...

Advanced Datadog logs: indexes, exclusion filters, Flex Logs, archives and Sensitive Data Scanner

Control the cost and compliance of Datadog logs: pipelines and processors, indexes and retention,...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!