RousselTMLEARNING

Observability

VictoriaMetrics best practices

Production checklist for VictoriaMetrics: sizing, retention, high availability, vmagent, cardinality, backups and monitoring.

1. Architecture #

Keeping it simple.

Principles #

VictoriaMetrics cluster reference architecture
  • Single node as long as possible; cluster when volume or multi-tenancy requires it.
  • vmagent for collection, with a disk buffer and two destinations for HA.
  • SSD disks, at least 20% free space (part merging).
  • Retention set explicitly, daily backups verified by a test restore.

2. Data #

Controlling cardinality.

Rules #

  • Cardinality explorer checked every month.
  • Relabeling in vmagent to drop useless labels and metrics.
  • Stream aggregation for metrics where only the total is useful.
  • Dashboards in standard PromQL if they must stay portable.

3. Operations #

Monitoring and upgrading.

Rituals #

  • Official dashboards and alerts installed.
  • Regular upgrades (changelog read, one instance at a time).
  • vmalert with -remoteRead.url to keep alert state.
  • Access protected by vmauth or a reverse proxy.

Reference diagram and quick check #

Checklist.

Checklist #

  1. Retention and disk sized
  2. vmagent with disk buffer
  3. vmalert connected to Alertmanager
  4. vmbackup backups and restore test
  5. Monitoring with the official dashboards
  6. Authenticated access

Related articles

VictoriaMetrics: why and how, from single node to cluster

What VictoriaMetrics brings compared with Prometheus alone, its components (single-node, vmagent, vmalert, vm…

Ingesting metrics into VictoriaMetrics: scraping, vmagent and remote write

All the ways to feed VictoriaMetrics: remote write from Prometheus, built-in scraping, vmagent and its disk b…

MetricsQL: PromQL with superpowers

The differences between MetricsQL and PromQL: implicit windows, extra functions (range, rollup, label), WITH …

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!