1. Archiving to object storage #
The basis for replay.
A well-partitioned S3 destination #
Bucket: telemetry-archive
Key prefix: cribl
Partitioning expr.: `${C.Time.strftime(_time ? _time : Date.now()/1000, '%Y/%m/%d/%H')}/${sourcetype}/${host}`
Data format: JSON (or Parquet)
Compression: gzip
Max file size: 32 MB · Max file open time: 300 s- Partitioning by date then type lets you replay one hour of one log type without reading everything else.
- Parquet (columnar) compresses better and can be queried directly by Cribl Search, Athena or ClickHouse; gzip JSON is the simplest to read back.
- A bucket lifecycle rule (move to cold class, expiry) sets retention and cost.
2. Replaying #
Re-injecting a specific period.
Collector and replay #
- Create an S3 collector pointing at the archive, with a path matching the partitioning:
/cribl/${_time:%Y}/${_time:%m}/${_time:%d}/${_time:%H}/${sourcetype}/${host}/. - Run it on demand (Run) with the time range and a filter (
host=="srv-42"): Cribl only reads the matching partitions thanks to the path variables. - Replayed events go through routes like any others: add a field (
replayed=true) to route them to the SIEM and tell them apart. - The collector Preview mode shows a sample before running the full collection.
Best practice : Also schedule regular collections (Schedule) for APIs that do not push their data: SaaS audit, inventories, cloud metrics.
3. Cribl Lake and Cribl Search #
Storing and querying without re-ingesting.
Cribl Lake #
Cribl Lake is managed storage in Cribl.Cloud: you create datasets with their retention, and a Cribl Lake destination writes to them from Stream or Edge, without managing a bucket or format.
Cribl Search #
Cribl Search queries data where it lives: Cribl Lake, S3 buckets and some APIs, with a Kusto-like language (KQL). Since 4.17, a Cribl Search destination also lets you send data to it directly from Stream or Edge.
dataset="archive_fw" action="deny"
| summarize count() by src_ip
| sort by count_ desc
| limit 10Good to know : The exact syntax and available datasets depend on your Cribl.Cloud workspace: start from the sample queries offered in the UI.
When to use what #
| Need | Solution |
|---|---|
| Keep everything at the lowest cost | S3 archive + lifecycle rules |
| Re-inject a period into the SIEM | S3 collector and replay |
| Find a needle without re-ingesting | Cribl Search |
| Managed storage, no bucket to administer | Cribl Lake |
Commands to know #
Commands to know
C.Time.strftime(_time, '%Y/%m/%d/%H'): partition by hour/${_time:%Y}/${_time:%m}/${_time:%d}/: collector path variablesaws s3 ls s3://archive/cribl/2026/11/17/ --recursive | head: check the archivedataset="…" | summarize count() by host: Cribl Search query
Sources #
Checked in November 2026 (Cribl 4.20): Cribl ships a minor release about every two months, check the release notes before relying on a detail.
My progress
Sign in to track your progress and take notes. Sign in
💬 Questions about this lesson
No questions yet: be the first to ask for help or share your configuration.
Recommended Articles
Bindplane: managing your OpenTelemetry telemetry pipelines
Understand Bindplane: a control plane for OpenTelemetry collectors via OpAMP, BDOT Collector, con...
OpenTelemetry Collector, Grafana Alloy and Bindplane: comparing three approaches to telemetry pipelines
Detailed comparison of three ways to manage telemetry pipelines: the OpenTelemetry Collector in Y...
OpenTelemetry Collector: connectors, extensions and observing the Collector
Go further with the Collector: connectors (spanmetrics, servicegraph, count, routing, failover), ...
RUM and Synthetic Monitoring with Datadog: synthetic transaction monitoring and user experience
Measure user experience with Datadog: browser and mobile RUM, Core Web Vitals, Session Replay, li...
Installing Bindplane: server, BDOT Collector and OpAMP connection
Install the Bindplane server (Docker Compose, Linux, Kubernetes), connect BDOT collectors via OpA...
OpenTelemetry Collector: processors (transforming, filtering and protecting telemetry)
Master OpenTelemetry Collector processors: batch, memory_limiter, attributes, resource, filter, t...
SigNoz, ClickStack and ClickHouse best practices for observability
Best practices for observability on ClickHouse with SigNoz or ClickStack: OpenTelemetry instrumen...
Dynatrace: GDPR and data privacy
Learn how to configure Dynatrace to comply with the GDPR, mask sensitive data and enable the coll...
Bindplane best practices
Checklist for operating Bindplane: organizing by labels and fleets, measured reduction, masking a...