[TP] My second pipeline




What you will learn in this TP :

Prerequisites

You must have done the lab 'My first pipeline'

Inputs configuration

You must create a file in /etc/logstash/conf.d named "0-input-file-advanced.conf".
  1. Reading a file
    In this file, you must add the logstash configuration to read the contents of all log files starting with: "/var/log/rousseltm/advanced-".

    You must add a type for this file with the value 'advanced'.

Filters configuration

You must create a second file named "101-filter-advances.conf".
  1. Data extraction
    You must identify the most suitable filter for a flat file and put the configuration that will extract the fields from the previously read log files knowing that these files are made up of several log formats.
  2. Type modification
    We want to convert the "HTTP_CODE" field to an integer. Indeed, there are often arithmetic operations on this field when making restitutions (dashboard, alerts...). For example, display all codes between 200 and 205.
  3. Field deletion
    We will remove the message field as we have already extracted the information that interests us.

Outputs configuration

You must modify the file "200-output-file-nginx-access".
  1. Writing to a file
    In this file, you must update the logstash configuration so that it writes the logs to the file that corresponds to its 'type'

Difficulty level: (3/5)

Recommended Articles

License consumption types

Understand the evolution of billing in Dynatrace: the difference between the ...

Grafana Alloy: The importance of Self-Monitoring

Discover why and how to configure Grafana Alloy so that it monitors itself, c...

Grafana Alloy: Understanding and exploiting the User Interface (UI)

Discover how to enable, secure, and use Grafana Alloy's built-in web interfac...

Grafana Alloy: Introduction and Architecture

Discover the fundamental concepts of Grafana Alloy, the transition from the s...

Grafana Alloy: Syntax and Configuration (Alloy Language)

Master Grafana Alloy's declarative syntax (River language), learn to manipula...

Grafana Alloy: Metrics Collection (Prometheus & Ecosystem)

Learn how to configure Grafana Alloy to collect, transform, and forward metri...

Grafana Alloy: Log Management with Loki

Discover how to configure Grafana Alloy to read log files, journald, or netwo...

Grafana Alloy: Trace Management with Tempo

Dive into distributed trace processing. Learn how to ingest OTLP, Jaeger, or ...

Grafana Alloy: Continuous Profiling with Pyroscope

Discover how to configure continuous profiling in your environments using Gra...

Grafana Alloy: Advanced Deployment and Clustering

Learn how to manage large-scale Grafana Alloy deployments. Configure Clusteri...

Grafana Assistant: AI at the service of observability

Discover Grafana Assistant, the artificial intelligence integrated into Grafa...

Grafana Alloy vs Dynatrace ActiveGate: Which one to choose?

Comparison between Grafana Alloy and Dynatrace ActiveGate. Understand the fun...