Lab: Detect anomalies with machine learning

What you will learn in this TP

• Load a dataset
• Create a job and its datafeed
• Read anomalies
• Forecast and explain a spike

Introduction #

What you will learn in this section

• Load a dataset
• Create a job and its datafeed
• Read anomalies
• Forecast and explain a spike

Context. Site traffic varies by hour and day: a fixed threshold fires too often or not enough. You let machine learning learn the normal rhythm and flag deviations.

Prerequisites: Docker, 4 GB of free RAM, jq; trial licence (provided by start-local)  ·  Estimated time: 40 min

Licence: this lab uses the 30-day trial (Platinum features) enabled automatically by start-local. On an existing cluster, enable it with POST _license/start_trial?acknowledge=true (once per major version); when it expires, the cluster falls back to Basic and jobs stop. The Data Visualizer stays free.

Hands-on #

Guided steps, with their solution.

1. 1. The lab

   Why? start-local starts Elasticsearch and Kibana with a 30-day trial licence, required for machine learning.

2. 2. The data

   Load the Kibana "web logs" sample dataset.

3. 3. The job and datafeed

   Why? The count detector models request volume per 15-minute bucket; geo.src is an influencer to explain deviations.

4. 4. Read the anomalies

   List the strongest anomalies, then open Anomaly Explorer in Kibana (Machine Learning › Anomaly Detection).

5. 5. Forecast and explain

   Run a 24-hour forecast, then use Log rate analysis (Machine Learning › AIOps Labs) on a spike in the dataset.

Difficulty level : ●●●○○ (3/5)