RousselTMLEARNING

Observability

Sources, destinations and routes in Cribl Stream

Receiving and sending data with Cribl Stream: push, pull, collector and internal sources, event breaking, destinations and output router, routing table, JavaScript filters, the Final option, QuickConnect and live data capture.

1. Sources #

Listen, collect, observe.

Source families #

FamilyExamplesHow it works
PushSyslog, Splunk HEC, Splunk TCP, HTTP, OpenTelemetry, Elasticsearch API, TCP JSONCribl listens on a port; senders push
PullKafka, Amazon SQS et S3, Azure Event Hubs, Google Pub/Sub, Office 365, OktaCribl fetches continuously
CollectorsS3, REST, Azure Blob, Google Cloud Storage, scriptscheduled or on-demand collection (replay)
InternalDatagen, Cribl Internal (logs et métriques), Cribl HTTP / Cribl TCPgenerated data, Cribl telemetry, flows from Edge or another Stream

Breaking into events #

Breaking

Sources receiving a byte stream (TCP, files, S3 objects) use Event Breakers: rules that split the stream into events (line, JSON, CSV, multi-line header) and extract the timestamp. A wrong rule produces truncated events or a wrong _time: it is the first setting to check.

Best practice : Add fields at the source (Fields), for example sourcetype, env or team: routes and pipelines then rely on them.

2. Destinations #

Where data goes.

Overview #

TypeExamples
SIEMSplunk (HEC, S2S, load-balanced), Microsoft Sentinel, Google SecOps, Elastic Security, QRadar
ObservabilityDatadog, Elasticsearch, Grafana Loki, Prometheus, OpenTelemetry, New Relic, Dynatrace
StorageAmazon S3 and compatible (MinIO), Azure Blob, Google Cloud Storage, Cribl Lake, filesystem
StreamingKafka, Amazon Kinesis, Azure Event Hubs, Google Pub/Sub
SpecialOutput Router (picks a destination by filter), Default, DevNull (discards), Cribl HTTP / TCP, Cribl Search

Each destination defines a format (JSON, raw, Parquet…), compression, TLS, and a backpressure behaviour: block, drop or persistent queue (see the production chapter).

3. Routes #

Who goes where, with which processing.

The routing table #

Routing table
  • Routes are evaluated in order, top to bottom. Each route has a filter (JavaScript expression), a pipeline and a destination.
  • With Final enabled (the default), an event matched by the route stops there. With Final disabled, a copy is processed and the event continues to the following routes: that is how you send the same data to several destinations.
  • The last route, true, catches the rest: send it to a default destination or to DevNull, deliberately.

Useful filters #

FilterMatches
sourcetype=='pan:traffic'Palo Alto traffic logs
__inputId.startsWith('syslog:')everything arriving through a syslog source
host.endsWith('.prod.local') && level=='error'errors from production hosts
/failed password/i.test(_raw)SSH authentication failures
C.Net.cidrMatch('10.0.0.0/8', src_ip)internal addresses

QuickConnect #

QuickConnect graphically links a source to a destination, with a pipeline or Pack in between, without the routing table. Handy for simple, independent flows; as soon as you need cloning, priorities or fine filtering, prefer routes.

4. Seeing the data #

Capture before transforming.

Capture and samples #

  • Live Data (on a source, route or destination) shows passing events in real time.
  • Capture records a few seconds or a few hundred events, at a precise point in the journey (before or after pipelines).
  • Save the capture as a sample file: it will be used to build and test pipelines in the preview.
Watch out : Samples contain real data: mask them or limit sharing if they contain personal data.

Commands to know #

Commands to know
  • logger -n 127.0.0.1 -P 9514 -T 'test cribl' : send a test syslog message
  • curl -s http://localhost:8088/services/collector/event -H "Authorization: Splunk <jeton>" -d '{"event":"test"}' : send an HEC event
  • __inputId.startsWith('syslog:') : filter by source
  • C.Net.cidrMatch('10.0.0.0/8', src_ip) : filter by network

Sources #

Checked in November 2026 (Cribl 4.20): Cribl ships a minor release about every two months, check the release notes before relying on a detail.

Related articles

Cribl: the data pipeline for observability and security

Discover Cribl: why a telemetry data pipeline, the Cribl suite (Stream, Edge, Lake, Search, Cribl.Cloud), the…

Installing and administering Cribl Stream

Installing Cribl Stream with Docker or on Linux, running it as a service, understanding $CRIBL_HOME, default …

Pipelines and functions: parse, enrich, mask

Building Cribl Stream pipelines: pre-processing, processing and post-processing, the function catalogue (Pars…

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!