OpenTelemetry Collector: exporters (sending telemetry to backends)

1. Overview of exporters #

Which exporter for which backend?

The most common ones #

2. Configuring exporters #

Complete examples.

OTLP, Prometheus and debug #

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls: { insecure: true }
  otlphttp/loki:
    endpoint: http://loki:3100/otlp
  prometheusremotewrite:
    endpoint: http://mimir:9009/api/v1/push
    headers: { X-Scope-OrgID: equipe-a }
    resource_to_telemetry_conversion: { enabled: true }
  prometheus:
    endpoint: 0.0.0.0:8889          # à scraper par Prometheus
  debug:
    verbosity: basic

service:
  pipelines:
    traces:  { receivers: [otlp], processors: [batch], exporters: [otlp/tempo, debug] }
    metrics: { receivers: [otlp], processors: [batch], exporters: [prometheusremotewrite] }
    logs:    { receivers: [otlp], processors: [batch], exporters: [otlphttp/loki] }

TLS and authentication #

exporters:
  otlphttp/cloud:
    endpoint: https://otlp-gateway-prod-eu-west-0.grafana.net/otlp
    auth: { authenticator: basicauth/cloud }
  otlp/secure:
    endpoint: backend.interne:4317
    tls:
      ca_file: /etc/otel/ca.pem
      cert_file: /etc/otel/client.pem
      key_file: /etc/otel/client.key

extensions:
  basicauth/cloud:
    client_auth: { username: ${env:GC_USER}, password: ${env:GC_TOKEN} }
service:
  extensions: [basicauth/cloud]

Common mistake: Don't leave tls.insecure: true in production: telemetry can contain sensitive data. Use environment variables (${env:NOM}) for secrets.

3. Reliability: retry and queueing #

Not losing data when the backend is slow.

sending_queue and retry_on_failure #

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    sending_queue:
      enabled: true
      num_consumers: 10
      queue_size: 5000
      storage: file_storage/queue       # file d'attente persistante sur disque
    retry_on_failure:
      enabled: true
      initial_interval: 5s
      max_interval: 30s
      max_elapsed_time: 5m
    timeout: 15s

extensions:
  file_storage/queue:
    directory: /var/lib/otelcol/queue

Good practice: Monitor otelcol_exporter_queue_size and otelcol_exporter_send_failed_spans: a filling queue signals a backend that is too slow, before data is lost.

4. Load balancing and troubleshooting #

loadbalancing and debug.

Load balancing by TraceID #

# Couche 1 : distribue par TraceID vers les Collectors de la couche 2 (tail sampling)
exporters:
  loadbalancing:
    routing_key: traceID
    protocol: { otlp: { tls: { insecure: true } } }
    resolver:
      dns: { hostname: otel-collector-tier2.monitoring.svc.cluster.local, port: 4317 }

Checking that data is sent #

curl -s localhost:8888/metrics | grep -E 'otelcol_exporter_(sent|send_failed)_(spans|metric_points|log_records)'
docker logs otelcol 2>&1 | grep -iE "error|dropping|retry" | tail
otelcol-contrib validate --config=config.yaml