RousselTMLEARNING

Article · Alloy: From Discovery to Expertise

Dispatching

3. Forwarding to Loki #

Secure forwarding to Grafana Loki.

The loki.write component #

loki.write "central" {
  external_labels = { cluster = "paris" }

  endpoint {
    url       = "https://loki.example.com/loki/api/v1/push"
    tenant_id = "equipe-paiement"

    batch_size = "1MiB"
    batch_wait = "1s"

    max_backoff_retries = 10
    max_backoff_period  = "5m"

    basic_auth {
      username = sys.env("LOKI_USER")
      password = sys.env("LOKI_PASSWORD")
    }
  }
}
SettingPurposeDefault
batch_sizeBatch size1MiB
batch_waitMaximum wait before sending1s
min_backoff_periodFirst delay before a retry500ms
max_backoff_periodMaximum delay between retries5m
max_backoff_retriesNumber of retries; 0 to retry forever10
tenant_idTenant, in multi-tenant mode–

What happens during an outage? #

Alloy retries with a growing delay. Once retries run out, the batch is lost and loki_write_dropped_entries_total increases. For file logs, the position protects what has not been read yet; for the rest, plan for the risk.

# Lines lost after retries ran out (alert on this)
rate(loki_write_dropped_entries_total[5m]) > 0

# Lines sent, retries
rate(loki_write_sent_entries_total[5m])
rate(loki_write_batch_retries_total[5m])

Several destinations #

As for metrics, forward_to accepts several receivers: one loki.write for current storage, another for audit, with longer retention on the server side. external_labels are added when sending: keep them for what identifies the site, such as cluster.

Checking arrival #

We checked sending directly in Loki:

curl -s http://localhost:3100/loki/api/v1/labels
curl -s -G http://localhost:3100/loki/api/v1/query_range \
  --data-urlencode 'query={job="app"}' \
  --data-urlencode "start=$(( $(date +%s)-600 ))000000000"
Best practice: keep few labels, with few values. job, env, level and service cover most needs; the rest is searched in the text at query time.

📄 Grafana Alloy: Log Management with Loki →

My progress

  • ⏱ Estimated time : ≈ 5 min

Sign in to track your progress and take notes. Sign in

⭐ Rate this lesson

4.0 initial rating from the teaching team

Sign in to rate this lesson →

💬 Recent discussions about this course

Ask a question about this lesson →See all discussions about this course

Course Glossary

Grafana Alloy

An open-source, vendor-neutral telemetry collector designed to replace Grafana Agent, fully compatible with OpenTelemetry (OTel).

Recommended Articles

Collecting logs into Loki: Alloy, Promtail, Docker and OpenTelemetry

Send logs to Loki: Grafana Alloy, Promtail, the Docker driver, OTLP, good labels, parsing and enr...

Grafana Alloy: collecting and transforming telemetry

Learn how to use Grafana Alloy to collect, transform and route logs, metrics and traces. Discover...

Grafana Alloy vs Grafana Agent vs Promtail: which one to choose?

Discover how telemetry collectors have evolved in the Grafana ecosystem. A detailed comparison of...

Grafana Alloy: collecting system metrics and local logs

Learn how to configure Grafana Alloy to monitor the server it runs on, with a focus on configurat...

Grafana Alloy components reference

A complete reference guide to all the components available in Grafana Alloy, organized by categor...

The LGTM stack (Loki, Grafana, Tempo, Mimir): architecture and setup

Grafana Labs' LGTM stack: Loki for logs, Grafana for visualization, Tempo for traces, Mimir for m...

Grafana Alloy: Trace Management with Tempo

Dive into distributed trace processing. Learn how to ingest OTLP, Jaeger, or Zipkin traces with G...

OpenTelemetry Collector, Grafana Alloy and Bindplane: comparing three approaches to telemetry pipelines

Detailed comparison of three ways to manage telemetry pipelines: the OpenTelemetry Collector in Y...

Grafana Alloy: The importance of Self-Monitoring

Discover why and how to configure Grafana Alloy so that it monitors itself, collecting its own lo...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!