Ingesting metrics into VictoriaMetrics: scraping, vmagent and remote write

1. From Prometheus #

Long-term storage in one line.

remote_write #

# prometheus.yml
remote_write:
  - url: http://vm:8428/api/v1/write
    queue_config: { max_samples_per_send: 10000, capacity: 20000, max_shards: 30 }

Prometheus keeps a short retention for its local alerts; VictoriaMetrics keeps the history.

2. Scraping directly #

Without Prometheus.

promscrape #

VictoriaMetrics and vmagent understand the Prometheus configuration file (scrape_configs, service discovery, relabeling):

victoria-metrics -promscrape.config=/etc/prometheus/prometheus.yml -storageDataPath=/data
# ou, plus courant, avec vmagent
vmagent -promscrape.config=/etc/prometheus/prometheus.yml \
        -remoteWrite.url=http://vm:8428/api/v1/write

Targets and their status are shown on /targets; /service-discovery shows labels before and after relabeling.

3. vmagent #

A robust collector.

Features #

vmagent \
  -promscrape.config=/etc/vmagent/scrape.yml \
  -remoteWrite.url=http://vm-eu:8428/api/v1/write \
  -remoteWrite.url=http://vm-us:8428/api/v1/write \
  -remoteWrite.tmpDataPath=/var/lib/vmagent \
  -remoteWrite.maxDiskUsagePerURL=10GB

Good practice: vmagent often replaces Prometheus in agent mode: less memory, a persistent disk buffer and native replication to several stores.

Stream aggregation #

# stream-aggr.yml : remplacer les séries par instance par un total par service
- match: 'http_requests_total'
  interval: 1m
  by: [service, code]
  outputs: [total]

The result (http_requests_total:1m_by_code_service_total) is sent instead of, or in addition to, the raw series: useful for very high-cardinality metrics where only the aggregate is consulted.

4. Other protocols #

A universal entry point.

Examples #

# InfluxDB line protocol
curl -d 'cpu,host=web-1 usage=42.5' http://vm:8428/write
# Import au format Prometheus texte
curl -d 'temperature{salle="serveur"} 21.5' http://vm:8428/api/v1/import/prometheus
# OpenTelemetry (OTLP HTTP, métriques)
#   http://vm:8428/opentelemetry/v1/metrics
# Export pour sauvegarde ou migration
curl -s 'http://vm:8428/api/v1/export?match[]={__name__="temperature"}'