[TP] Lab: Analyse logs on the command line

What you will learn in this TP
Navigate and inspect files
Combine awk, sort and uniq in a pipe
Redirect output and archive

Introduction #

What you will learn in this section
Navigate and inspect files
Combine awk, sort and uniq in a pipe
Redirect output and archive

Context. The web team reports intermittent errors. Before installing an analytics tool, you dig into the access log with standard shell tools.

Prerequisites: a disposable Ubuntu 24.04 virtual machine (Multipass, Vagrant or cloud) with a sudo account · Estimated time: 40 min

An error report and a timestamped archive.

Guided steps, with their solution.

1. Prepare the workspace
Why? A realistic dataset lets you practise without touching system logs.
Create ~/tp-shell/{logs,rapports} and generate 500 nginx-style lines in logs/access.log.
2. Inspect the file
Why? Before analysing, check the size, type and line format.
Show the file size and type, the last 5 lines, then browse it with less searching for 500.
3. Count HTTP status codes
Why? The 9th field holds the status code: sort | uniq -c is the universal counting tool.
Show the number of requests per HTTP code, most frequent first.
4. Find the error sources
Why? Filtering before counting answers a precise question.
List the 5 IP addresses and the URLs producing the most 500 errors, and the total of 4xx and 5xx errors.
5. Compute the volume served
Why? awk can sum a column: handy for volumes, durations or amounts.
Compute the total bytes served (10th field) in kilobytes, then the volume per URL.
6. Produce and archive a report
Why? Redirections turn a one-off analysis into a reusable deliverable.
Write a dated report in rapports/ (HTTP codes and top 500 errors), display it with tee, then archive the folder.
7. Search the system
Why? find and grep are the go-to tools to locate a configuration or a large file.
Find files in /etc mentioning PermitRootLogin, then files over 50 MB on the root partition.

Difficulty level : ●○○○○ (1/5)