RousselTMLEARNING

TP · Cribl: from discovery to expertise

Lab: parsing, enriching and masking Apache logs

What you will learn in this TP
  • Configure an HEC source
  • Extract fields with Regex Extract
  • Enrich with Lookup
  • Mask with Mask
  • Test in the preview and wire the route

Introduction #

What you will learn in this section
  • Configure an HEC source
  • Extract fields with Regex Extract
  • Enrich with Lookup
  • Mask with Mask
  • Test in the preview and wire the route

Context. The shop's access logs contain customer e-mail addresses in URLs. The SIEM must no longer receive them in clear, and analysts want a readable category for each HTTP code.

Prerequisites: a machine with Docker and Docker Compose (4 GB free memory), curl and jq, the Lab 1 instance  ·  Estimated time: 50 min

A pipeline turning a raw Apache line into a structured, enriched event with no personal data in clear.

Setup #

Guided steps, with their solution.

  1. 1. HEC source and data

    Why? HEC is Splunk's HTTP protocol, widely used by applications.

    Create a Splunk HEC source in_hec on port 8088 with the token tp2-hec-token, then send 100 Apache lines.
  2. 2. Capture a sample

    Why? The pipeline is built on real lines.

    Capture 100 events from in_hec and save them as the sample apache_access.log. Create the pl_apache pipeline and load the sample in the preview.
  3. 3. Extract fields

    Why? Structured fields allow filtering, aggregating and routing.

    Add a Regex Extract function on _raw creating client, method, uri, status and bytes, then Numerify to convert status and bytes to numbers.
  4. 4. Enrich

    Why? A readable category saves every analyst from memorizing HTTP codes.

    Create the http_status.csv table in Processing › Knowledge › Lookups, then a Lookup function adding status_category.
  5. 5. Mask e-mails

    Why? The hash still allows counting distinct customers, without revealing their address.

    Add a Mask function on _raw and uri replacing the email= value with its SHA-256, then an Eval adding env='prod'.
  6. 6. Wire and check

    Why? The preview proves the logic, the output proves the wiring.

    Check in the preview (OUT) that no address remains in clear. Add an apache route (filter sourcetype=='apache:access', pipeline pl_apache, destination out_files), resend data and check the files.

Difficulty level : ●●●○○ (3/5)

My progress

  • ⏱ Estimated time : ≈ 50 min

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

Dynatrace: GDPR and data privacy

Learn how to configure Dynatrace to comply with the GDPR, mask sensitive data and enable the coll...

Grafana Alloy: Log Management with Loki

Discover how to configure Grafana Alloy to read log files, journald, or network streams, process ...

Cribl: the data pipeline for observability and security

Discover Cribl: why a telemetry data pipeline, the Cribl suite (Stream, Edge, Lake, Search, Cribl...

Advanced Datadog logs: indexes, exclusion filters, Flex Logs, archives and Sensitive Data Scanner

Control the cost and compliance of Datadog logs: pipelines and processors, indexes and retention,...

Bindplane: managing your OpenTelemetry telemetry pipelines

Understand Bindplane: a control plane for OpenTelemetry collectors via OpAMP, BDOT Collector, con...

OpenTelemetry Collector, Grafana Alloy and Bindplane: comparing three approaches to telemetry pipelines

Detailed comparison of three ways to manage telemetry pipelines: the OpenTelemetry Collector in Y...

Logs with OpenTelemetry: data model, appenders, correlation and collection

Master the log signal: the LogRecord data model, the Logs Bridge API and appenders (Logback, Log4...

Archiving, replaying and searching: S3, Replay, Cribl Lake and Cribl Search

Keeping all data cheaply and finding it on demand: partitioned S3 destination, JSON and Parquet f...

Cribl Stream in production: distributed architecture, persistent queues and monitoring

Taking Cribl Stream to production: Leader, Worker Groups and processes, commit and deploy, scalin...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!