1. The components #
An architecture borrowed from Mimir.
Overview #
Available in: ✓ OSS ✓ Enterprise ✕ Cloud In Grafana Cloud the service is managed by Grafana Labs: nothing to install
| Component | Role |
|---|---|
| distributor | Receives profiles, validates, applies limits, distributes to the ingesters |
| ingester | Aggregates recent profiles in memory and writes blocks to object storage |
| querier / query-frontend | Merges the requested profiles and builds flame graphs and series |
| store-gateway | Reads historical blocks from object storage |
| compactor | Merges blocks and applies retention |
Good to know: The internal architecture evolves quickly between versions (notably towards ingestion written directly to object storage). The principles (object storage, per-component scaling, Grafana compatibility) remain the same: refer to the release notes of your version.
2. Running Pyroscope #
Monolithic to start with.
Docker #
docker run -d --name pyroscope -p 4040:4040 grafana/pyroscope:latest
curl -s localhost:4040/readyThe built-in web interface is available at http://localhost:4040; so is the ingestion API (SDK: /ingest, Alloy: /push.v1.PusherService/Push).
Configuration with object storage #
# pyroscope.yaml
target: all
server:
http_listen_port: 4040
storage:
backend: s3
s3:
endpoint: minio:9000
bucket_name: pyroscope
access_key_id: pyroscope
secret_access_key: supersecret
insecure: true
limits:
compactor_blocks_retention_period: 30d
ingestion_rate_mb: 4
ingestion_burst_size_mb: 8
max_label_names_per_series: 30pyroscope -config.file=/etc/pyroscope.yamlGood practice: Without object storage, Pyroscope writes to the local disk (filesystem): perfect for a lab, not for production.3. Grafana and profilecli #
Explore and automate.
Data source #
In Grafana, add a Grafana Pyroscope data source with the URL http://pyroscope:4040. The Profiles Drilldown app (formerly Explore Profiles) offers query-less navigation: services, profile types, labels, then flame graph.
profilecli #
# Lister les séries et les types de profils disponibles
profilecli query series --url http://localhost:4040 --query '{service_name="checkout"}'
# Télécharger un profil fusionné au format pprof (analysable avec go tool pprof)
profilecli query profile --url http://localhost:4040 \
--profile-type process_cpu:cpu:nanoseconds:cpu:nanoseconds \
--query '{service_name="checkout"}' --from now-1h --output pprof=cpu.pb.gz
# Envoyer un profil pprof existant
profilecli upload --url http://localhost:4040 --extra-labels service_name=batch cpu.pb.gz