1. Requirements #
What is expected.
Context #
An operations team wants to centralise logs from its Linux servers and from a legacy application writing to a text file, keep them 30 days, analyse them and be warned of error bursts.
| Requirement | Component | Course |
|---|---|---|
| Collect server system logs | Elastic Agent + Fleet | Elastic Agent |
| Structure the legacy application logs | Logstash (dissect, translate) | Logstash |
| Keep 30 days | Data streams + ILM | Elasticsearch |
| Analyse and visualise | ES|QL, Lens | Kibana |
| Alert on error bursts | ES|QL rule + connector | Kibana |
| Read access for the team | Dedicated role and space | Elasticsearch, Kibana |
2. Steps #
The guided lab.
Outline #
- Set up the Fleet lab (lab 1 of the Elastic Agent course).
- Check system log collection.
- Create the index template and the 30-day ILM policy.
- Add Logstash for the legacy application.
- Analyse with ES|QL and build the dashboard.
- Create the alert rule and the read-only role.
3. Assessment grid #
What validates the project.
Criteria #
| Criterion | Check |
|---|---|
| Collection | the agent is Healthy in Fleet and logs-system.*-lab receives data |
| Structuring | the log.level, http.response.status_code fields and the translated label exist in logs-legacyapp-lab |
| Retention | GET logs-legacyapp-lab/_ilm/explain shows the 30-day policy |
| Analysis | a dashboard with at least three visualisations across both sources |
| Alerting | the rule goes Active during a simulated error burst |
| Security | a "reader" user sees the dashboard but cannot change anything |
Good practice: Export the dashboard (saved objects) and version it with your configuration files: that is your deliverable.
My progress
Sign in to track your progress and take notes. Sign in
💬 Questions about this lesson
No questions yet: be the first to ask for help or share your configuration.
Recommended Articles
Machine learning in Elastic: anomaly detection and AIOps
Automatically detect abnormal behaviour: anomaly detection jobs, datafeeds, detectors, influencer...
Elastic Security: SIEM, detection rules and investigation
Use Elastic Security as a SIEM: ECS data sources, detection rule types (query, EQL, ES|QL, thresh...
Kibana Maps and Canvas: geographic maps and presentations
Build maps with Kibana Maps (document, aggregation and choropleth layers, ES|QL) and dynamic pres...
Transforms and enrich: summarising and enriching data in Elasticsearch
Build entity-centric indices with pivot and latest transforms, and enrich documents at ingest tim...
Logstash in production: persistent queues, DLQ and multiple pipelines
Make Logstash reliable in production: persistent queue, dead letter queue, multiple pipelines and...
Logstash: enrich, test, monitor and deploy on Kubernetes
Enrich with translate, geoip and elasticsearch, consume Kafka, test pipelines, monitor Logstash a...
Elastic Observability: logs, infrastructure, APM and synthetics
Elastic Observability features: log exploration, infrastructure inventory with Elastic Agent and ...
Alerts, connectors and SLOs in Kibana
Create alert rules in Kibana (Elasticsearch query, ES|QL, custom threshold, logs), configure conn...
Kibana and Elastic Observability best practices
Checklist for running Kibana in production: versions, security, spaces and roles, versioned dashb...