Lab: Install OpenSearch, index and search with correction

What you will learn in this TP

• Start OpenSearch
• Declare types
• Search and aggregate

Introduction #

What you will learn in this section

• Start OpenSearch
• Declare types
• Search and aggregate

Context. Your team wants an Apache 2.0 log search engine. You evaluate OpenSearch on a prototype.

Prerequisites: Docker Compose, 4 GB of free memory  ·  Estimated time: 45 min

A node and its console.

Setup #

Guided steps, with their solution.

1. 1. Start the stack

   Why? Without a strong password, OpenSearch refuses to start.

   Write .env and the Compose file, set vm.max_map_count, then check cluster state.
   See the correction

   OPENSEARCH_INITIAL_ADMIN_PASSWORD=Str0ng!Passw0rd-2026

   services:
     opensearch:
       image: opensearchproject/opensearch:3
       environment:
         discovery.type: single-node
         OPENSEARCH_INITIAL_ADMIN_PASSWORD: ${OPENSEARCH_INITIAL_ADMIN_PASSWORD}
         OPENSEARCH_JAVA_OPTS: -Xms1g -Xmx1g
         path.repo: /usr/share/opensearch/data/snapshots
       ports:
         - "9200:9200"
       volumes:
         - os-data:/usr/share/opensearch/data
   
     dashboards:
       image: opensearchproject/opensearch-dashboards:3
       environment:
         OPENSEARCH_HOSTS: '["https://opensearch:9200"]'
       ports:
         - "5601:5601"
       depends_on: [opensearch]
   
   volumes:
     os-data:

   sudo sysctl -w vm.max_map_count=262144
   docker compose up -d
   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/json' https://localhost:9200/_cat/health?v

2. 2. Declare an index template

   Why? A field type cannot change after the first write.

   Create a template for logs-app-*.
   See the correction

   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/json' -X PUT https://localhost:9200/_index_template/logs_app -d '{
     "index_patterns": ["logs-app-*"],
     "template": {
       "settings": {"number_of_shards": 1, "number_of_replicas": 0},
       "mappings": {"properties": {
         "@timestamp": {"type": "date"},
         "level": {"type": "keyword"},
         "service": {"type": "keyword"},
         "message": {"type": "text"},
         "duree_ms": {"type": "integer"}
       }}
     }
   }'

3. 3. Index in batches

   Why? _bulk is the normal path for data.

   Index three documents, then read the errors field of the response.
   See the correction

   cat > lot.ndjson <<'EOF'
   {"index":{"_index":"logs-app-000001"}}
   {"@timestamp":"2026-11-04T10:00:00Z","level":"ERROR","service":"paiement","message":"timeout vers la banque utilisateur=alice","duree_ms":3100}
   {"index":{"_index":"logs-app-000001"}}
   {"@timestamp":"2026-11-04T10:00:01Z","level":"INFO","service":"paiement","message":"paiement accepté","duree_ms":120}
   {"index":{"_index":"logs-app-000001"}}
   {"@timestamp":"2026-11-04T10:00:02Z","level":"INFO","service":"panier","message":"article ajouté","duree_ms":40}
   EOF
   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/x-ndjson' \
     -X POST 'https://localhost:9200/_bulk?refresh=true' --data-binary @lot.ndjson

4. 4. Search and aggregate

   Why? filter is faster than must when the score does not matter.

   Search “banque” in the paiement service, then compute the p95 duration per service.
   See the correction

   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/json' -X POST 'https://localhost:9200/logs-app-*/_search' -d '{
     "query": {"bool": {
       "must":   [{"match": {"message": "banque"}}],
       "filter": [{"term": {"service": "paiement"}}]
     }}
   }'

   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/json' -X POST 'https://localhost:9200/logs-app-*/_search' -d '{
     "size": 0,
     "aggs": {"par_service": {
       "terms": {"field": "service"},
       "aggs": {"p95": {"percentiles": {"field": "duree_ms", "percents": [95]}}}
     }}
   }'

5. 5. Try PPL and the console

   Why? PPL is very readable for exploring logs.

   Count documents per level with PPL, then find them in Discover at http://localhost:5601.
   See the correction

   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/json' -X POST https://localhost:9200/_plugins/_ppl \
     -d '{"query":"source=logs-app-* | where service=\"paiement\" | stats count() by level"}'
   
   curl -sk -u 'admin:Str0ng!Passw0rd-2026' -H 'Content-Type: application/json' -X POST https://localhost:9200/_plugins/_sql \
     -d '{"query":"SELECT service, COUNT(*) n FROM logs-app-000001 GROUP BY service"}'

Difficulty level : ●○○○○ (1/5)