1. Archiving to object storage #
The basis for replay.
A well-partitioned S3 destination #
Bucket: telemetry-archive
Key prefix: cribl
Partitioning expr.: `${C.Time.strftime(_time ? _time : Date.now()/1000, '%Y/%m/%d/%H')}/${sourcetype}/${host}`
Data format: JSON (or Parquet)
Compression: gzip
Max file size: 32 MB · Max file open time: 300 s- Partitioning by date then type lets you replay one hour of one log type without reading everything else.
- Parquet (columnar) compresses better and can be queried directly by Cribl Search, Athena or ClickHouse; gzip JSON is the simplest to read back.
- A bucket lifecycle rule (move to cold class, expiry) sets retention and cost.
2. Replaying #
Re-injecting a specific period.
Collector and replay #
- Create an S3 collector pointing at the archive, with a path matching the partitioning:
/cribl/${_time:%Y}/${_time:%m}/${_time:%d}/${_time:%H}/${sourcetype}/${host}/. - Run it on demand (Run) with the time range and a filter (
host=="srv-42"): Cribl only reads the matching partitions thanks to the path variables. - Replayed events go through routes like any others: add a field (
replayed=true) to route them to the SIEM and tell them apart. - The collector Preview mode shows a sample before running the full collection.
Best practice : Also schedule regular collections (Schedule) for APIs that do not push their data: SaaS audit, inventories, cloud metrics.
3. Cribl Lake and Cribl Search #
Storing and querying without re-ingesting.
Cribl Lake #
Cribl Lake is managed storage in Cribl.Cloud: you create datasets with their retention, and a Cribl Lake destination writes to them from Stream or Edge, without managing a bucket or format.
Cribl Search #
Cribl Search queries data where it lives: Cribl Lake, S3 buckets and some APIs, with a Kusto-like language (KQL). Since 4.17, a Cribl Search destination also lets you send data to it directly from Stream or Edge.
dataset="archive_fw" action="deny"
| summarize count() by src_ip
| sort by count_ desc
| limit 10Good to know : The exact syntax and available datasets depend on your Cribl.Cloud workspace: start from the sample queries offered in the UI.
When to use what #
| Need | Solution |
|---|---|
| Keep everything at the lowest cost | S3 archive + lifecycle rules |
| Re-inject a period into the SIEM | S3 collector and replay |
| Find a needle without re-ingesting | Cribl Search |
| Managed storage, no bucket to administer | Cribl Lake |
Commands to know #
Commands to know
C.Time.strftime(_time, '%Y/%m/%d/%H'): partition by hour/${_time:%Y}/${_time:%m}/${_time:%d}/: collector path variablesaws s3 ls s3://archive/cribl/2026/11/17/ --recursive | head: check the archivedataset="…" | summarize count() by host: Cribl Search query
Sources #
Checked in November 2026 (Cribl 4.20): Cribl ships a minor release about every two months, check the release notes before relying on a detail.
