RousselTMLEARNING

Article · Cribl: from discovery to expertise

Archiving, replaying and searching

1. Archiving to object storage #

The basis for replay.

A well-partitioned S3 destination #

Storing and querying
Bucket:               telemetry-archive
Key prefix:           cribl
Partitioning expr.:   `${C.Time.strftime(_time ? _time : Date.now()/1000, '%Y/%m/%d/%H')}/${sourcetype}/${host}`
Data format:          JSON   (or Parquet)
Compression:          gzip
Max file size:        32 MB   ·   Max file open time: 300 s
  • Partitioning by date then type lets you replay one hour of one log type without reading everything else.
  • Parquet (columnar) compresses better and can be queried directly by Cribl Search, Athena or ClickHouse; gzip JSON is the simplest to read back.
  • A bucket lifecycle rule (move to cold class, expiry) sets retention and cost.

2. Replaying #

Re-injecting a specific period.

Collector and replay #

Replay
  1. Create an S3 collector pointing at the archive, with a path matching the partitioning: /cribl/${_time:%Y}/${_time:%m}/${_time:%d}/${_time:%H}/${sourcetype}/${host}/.
  2. Run it on demand (Run) with the time range and a filter (host=="srv-42"): Cribl only reads the matching partitions thanks to the path variables.
  3. Replayed events go through routes like any others: add a field (replayed=true) to route them to the SIEM and tell them apart.
  4. The collector Preview mode shows a sample before running the full collection.
Best practice : Also schedule regular collections (Schedule) for APIs that do not push their data: SaaS audit, inventories, cloud metrics.

3. Cribl Lake and Cribl Search #

Storing and querying without re-ingesting.

Cribl Lake #

Cribl Lake is managed storage in Cribl.Cloud: you create datasets with their retention, and a Cribl Lake destination writes to them from Stream or Edge, without managing a bucket or format.

Cribl Search #

Cribl Search queries data where it lives: Cribl Lake, S3 buckets and some APIs, with a Kusto-like language (KQL). Since 4.17, a Cribl Search destination also lets you send data to it directly from Stream or Edge.

dataset="archive_fw" action="deny"
| summarize count() by src_ip
| sort by count_ desc
| limit 10
Good to know : The exact syntax and available datasets depend on your Cribl.Cloud workspace: start from the sample queries offered in the UI.

When to use what #

NeedSolution
Keep everything at the lowest costS3 archive + lifecycle rules
Re-inject a period into the SIEMS3 collector and replay
Find a needle without re-ingestingCribl Search
Managed storage, no bucket to administerCribl Lake

Commands to know #

Commands to know
  • C.Time.strftime(_time, '%Y/%m/%d/%H') : partition by hour
  • /${_time:%Y}/${_time:%m}/${_time:%d}/ : collector path variables
  • aws s3 ls s3://archive/cribl/2026/11/17/ --recursive | head : check the archive
  • dataset="…" | summarize count() by host : Cribl Search query

Sources #

Checked in November 2026 (Cribl 4.20): Cribl ships a minor release about every two months, check the release notes before relying on a detail.

📄 Archiving, replaying and searching: S3, Replay, Cribl Lake and Cribl Search →

My progress

  • ⏱ Estimated time : ≈ 5 min

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

Bindplane: managing your OpenTelemetry telemetry pipelines

Understand Bindplane: a control plane for OpenTelemetry collectors via OpAMP, BDOT Collector, con...

OpenTelemetry Collector, Grafana Alloy and Bindplane: comparing three approaches to telemetry pipelines

Detailed comparison of three ways to manage telemetry pipelines: the OpenTelemetry Collector in Y...

OpenTelemetry Collector: connectors, extensions and observing the Collector

Go further with the Collector: connectors (spanmetrics, servicegraph, count, routing, failover), ...

RUM and Synthetic Monitoring with Datadog: synthetic transaction monitoring and user experience

Measure user experience with Datadog: browser and mobile RUM, Core Web Vitals, Session Replay, li...

Installing Bindplane: server, BDOT Collector and OpAMP connection

Install the Bindplane server (Docker Compose, Linux, Kubernetes), connect BDOT collectors via OpA...

OpenTelemetry Collector: processors (transforming, filtering and protecting telemetry)

Master OpenTelemetry Collector processors: batch, memory_limiter, attributes, resource, filter, t...

SigNoz, ClickStack and ClickHouse best practices for observability

Best practices for observability on ClickHouse with SigNoz or ClickStack: OpenTelemetry instrumen...

Dynatrace: GDPR and data privacy

Learn how to configure Dynatrace to comply with the GDPR, mask sensitive data and enable the coll...

Bindplane best practices

Checklist for operating Bindplane: organizing by labels and fleets, measured reduction, masking a...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!