What you will learn in this TP
- Install Loki and Fluent Bit
- Enrich with metadata
- Exclude a pod by annotation
Introduction #
What you will learn in this section
- Install Loki and Fluent Bit
- Enrich with metadata
- Exclude a pod by annotation
Context. The cluster is growing and container logs can only be seen with kubectl logs. You centralise all logs along with their Kubernetes context.
Prerequisites: kind, kubectl, helm · Estimated time: 45 min
All the cluster logs, with namespace and pod.Setup #
Guided steps, with their solutions.
1. Create the cluster and Loki
Why? Loki in monolithic mode is enough for this lab.
Create a kind cluster and install Loki (grafana/lokichart in single binary mode).2. Install Fluent Bit
Why? The chart deploys the DaemonSet, the RBAC permissions (for the kubernetes filter) and the configuration.
Install Fluent Bit with a Loki output and the kubernetes filter.3. Deploy a chatty application
Why? You need logs to check the enrichment.
Start a pod that writes one line per second in thedemonamespace.4. Query with metadata
Why? Namespace and pod labels make searching immediate.
Open a tunnel to Loki and search for the logs of thedemonamespace.5. Exclude by annotation
Why? Teams manage the noise of their own pods without changing the central configuration.
Annotate the pod withfluentbit.io/exclude: "true"and check that its new lines no longer arrive.6. Clean up
Why? To free up resources.
Delete the cluster.
Difficulty level : ●●●○○ (3/5)
My progress
Sign in to track your progress and take notes. Sign in
Recommended Articles
Fluent Bit sur Kubernetes : DaemonSet, multiline, tampon et supervision
Déployer Fluent Bit sur Kubernetes avec Helm, enrichir avec le filtre kubernetes, gérer les logs ...
Fluent Bit : parsers et filters, structurer et transformer les logs
Structurer et transformer les logs dans Fluent Bit : parsers (json, regex, logfmt, ltsv), parsers...
Fluent Bit : pipeline, configuration et plugins essentiels
Les concepts de Fluent Bit (inputs, parsers, filters, tags et match, buffer, outputs), la configu...
Bonnes pratiques de collecte de logs avec Fluent Bit et Vector
Check-list de production pour la collecte de logs : structure, filtrage à la source, masquage, ta...
Collecteurs de logs légers : Fluent Bit, Vector et les autres
Pourquoi un collecteur de logs, ce qui distingue les agents légers (Fluent Bit, Vector) des colle...
Fluent Bit : les inputs (entrées) du pipeline en détail
Tout sur les inputs de Fluent Bit : rôle des tags, lecture de fichiers avec tail (positions, rota...
Fluent Bit : les outputs (sorties), routage et fiabilité de l'envoi
Envoyer les données de Fluent Bit : routage par Match et Match_Regex, sorties Loki, Elasticsearch...
Fluent Bit ou Vector ? Choisir, combiner et migrer
Comparer Fluent Bit et Vector (performances, transformations, écosystème, gouvernance), les combi...
Kubernetes : Pods, ReplicaSets et Deployments (déployer et mettre à jour)
Maîtriser les workloads Kubernetes : anatomie d'un pod, ReplicaSet, Deployment, stratégies Rollin...