1. Log sources #
Reading files, journald, and network receivers.
Where do logs come from? #
Available in: ✓ OSS ✓ Enterprise ✓ Cloud
| Component | Source |
|---|---|
loki.source.file | Files, with rotation handling |
loki.source.journal | systemd journal |
loki.source.docker | Docker containers |
loki.source.kubernetes | Pods, through the Kubernetes API |
loki.source.syslog | Syslog messages received over the network |
loki.source.api | HTTP reception, in Loki format |
otelcol.receiver.otlp | OpenTelemetry logs from applications |
Reading files #
local.file_match finds files, loki.source.file reads them. Target labels, here job, travel with each line; __path__ designates the file and becomes the filename label.
local.file_match "app" {
path_targets = [{ "__path__" = "/var/log/app/*.log", "job" = "app" }]
}
loki.source.file "app" {
targets = local.file_match.app.targets
forward_to = [loki.write.local.receiver]
}
loki.write "local" {
endpoint {
url = "http://loki:3100/loki/api/v1/push"
}
}Alloy remembers its position in --storage.path: at restart, it resumes where it stopped. If this folder is not kept, the whole file is re-read and Loki receives duplicates.
Nothing arrives? Check permissions first: thealloyuser must be able to read the files. On Debian, add it to theadmgroup for/var/log.
The systemd journal #
loki.source.journal "systeme" {
forward_to = [loki.write.local.receiver]
labels = { job = "journal" }
}
loki.write "local" {
endpoint {
url = "http://loki:3100/loki/api/v1/push"
}
}loki.source.journal can also transform internal journal fields with relabel_rules, for example to keep the unit name.
Checking without Loki #
To see what Alloy sees, plug loki.echo into forward_to: each line is written to Alloy's logs with its labels. The tests for this course relied on this component.
2. Processing pipelines #
Filtering, parsing (JSON, Regex), and relabeling.
Processing line by line #
loki.process applies stages, in order. They share a working area, the “extracted map”: one stage drops values there, a later one uses them.
loki.process "app" {
forward_to = [loki.write.local.receiver]
stage.json {
expressions = { level = "", service = "", trace_id = "", ts = "", msg = "" }
}
stage.drop {
source = "level"
value = "debug"
}
stage.timestamp {
source = "ts"
format = "RFC3339"
}
stage.labels {
values = { level = "", service = "" }
}
stage.structured_metadata {
values = { trace_id = "" }
}
stage.static_labels {
values = { env = "production" }
}
stage.output {
source = "msg"
}
}
loki.write "local" {
endpoint {
url = "http://loki:3100/loki/api/v1/push"
}
}What this pipeline produces #
Tested end to end with a real Loki, on this file:
{"ts":"2026-11-05T10:00:00Z","level":"error","service":"paiement","trace_id":"abc123","msg":"timeout banque"}
{"ts":"2026-11-05T10:00:00Z","level":"debug","service":"paiement","trace_id":"abc124","msg":"détail inutile"}
{"ts":"2026-11-05T10:00:00Z","level":"info","service":"panier","trace_id":"abc125","msg":"article ajouté"}
ligne non json| Line | In Loki |
|---|---|
error, paiement | Text timeout banque, labels level=error, service=paiement, env=production, metadata trace_id=abc123 |
debug | Discarded: loki_process_dropped_lines_total is 1 |
info, panier | Text article ajouté, labels level=info, service=panier |
ligne non json | Kept as is, with no level or service |
The most useful stages #
| Stage | Purpose |
|---|---|
stage.json, stage.regex, stage.logfmt | Extract values |
stage.labels | Turn an extracted value into a label |
stage.static_labels | Add a label with a fixed value |
stage.structured_metadata | Keep a varied value without creating streams |
stage.timestamp | Use the log time |
stage.drop | Discard lines |
stage.output | Replace the line text |
stage.multiline | Group several lines |
Do not confusestage.labelsandstage.static_labels. Instage.labels { values = { env = "production" } },productionis the name of an extracted value, not text.alloy validateaccepts it, but we checked that the label never appears. Usestage.static_labels.
Mind cardinality: each label combination creates a stream in Loki.trace_idor a user ID as a label quickly degrades performance: that is the role ofstage.structured_metadata.
Multi-line stack traces #
Without care, each line of a Java stack becomes a separate log. stage.multiline groups them: a new entry starts at each line matching firstline.
loki.process "java" {
forward_to = [loki.write.local.receiver]
stage.multiline {
firstline = "^\\d{4}-\\d{2}-\\d{2}"
max_wait_time = "3s"
}
stage.regex {
expression = "^\\S+ \\S+ (?P<level>[A-Z]+) "
}
stage.labels {
values = { level = "" }
}
}
loki.write "local" {
endpoint {
url = "http://loki:3100/loki/api/v1/push"
}
}Tested: the 3 stack lines form a single entry, with level=ERROR.
3. Forwarding to Loki #
Secure forwarding to Grafana Loki.
The loki.write component #
loki.write "central" {
external_labels = { cluster = "paris" }
endpoint {
url = "https://loki.example.com/loki/api/v1/push"
tenant_id = "equipe-paiement"
batch_size = "1MiB"
batch_wait = "1s"
max_backoff_retries = 10
max_backoff_period = "5m"
basic_auth {
username = sys.env("LOKI_USER")
password = sys.env("LOKI_PASSWORD")
}
}
}| Setting | Purpose | Default |
|---|---|---|
batch_size | Batch size | 1MiB |
batch_wait | Maximum wait before sending | 1s |
min_backoff_period | First delay before a retry | 500ms |
max_backoff_period | Maximum delay between retries | 5m |
max_backoff_retries | Number of retries; 0 to retry forever | 10 |
tenant_id | Tenant, in multi-tenant mode | – |
What happens during an outage? #
Alloy retries with a growing delay. Once retries run out, the batch is lost and loki_write_dropped_entries_total increases. For file logs, the position protects what has not been read yet; for the rest, plan for the risk.
# Lines lost after retries ran out (alert on this)
rate(loki_write_dropped_entries_total[5m]) > 0
# Lines sent, retries
rate(loki_write_sent_entries_total[5m])
rate(loki_write_batch_retries_total[5m])Several destinations #
As for metrics, forward_to accepts several receivers: one loki.write for current storage, another for audit, with longer retention on the server side. external_labels are added when sending: keep them for what identifies the site, such as cluster.
Checking arrival #
We checked sending directly in Loki:
curl -s http://localhost:3100/loki/api/v1/labels
curl -s -G http://localhost:3100/loki/api/v1/query_range \
--data-urlencode 'query={job="app"}' \
--data-urlencode "start=$(( $(date +%s)-600 ))000000000"Best practice: keep few labels, with few values.job,env,levelandservicecover most needs; the rest is searched in the text at query time.
2026 update: what has changed #
Recent changes worth knowing (checked in November 2026).
Migrating from Promtail #
Promtail reached end of life on March 2, 2026: no more fixes or support. Alloy provides a converter, but a few points need a manual check:
- The metrics exposed by Alloy differ from Promtail's: adapt alerts and dashboards.
- The positions file is stored elsewhere: plan how reading resumes so old logs are not sent again.
- For a gradual transition, Alloy can temporarily read a Promtail configuration (
--config.format=promtail).
alloy convert --source-format=promtail --report=report.txt promtail.yaml -o config.alloy
alloy run --config.format=promtail promtail.yaml # temporary transitionGood to know: Grafana Alloy ships a minor release roughly every three weeks; 1.21 (October 2026) is the latest at the time of this update. Read the release notes before each upgrade: some releases bring a new Prometheus or OpenTelemetry Collector branch, with flagged breaking changes.
