Continuous profiling with eBPF

1. The principle #

Sampling call stacks from the kernel.

How it works #

An eBPF program attached to a perf event (for example 19 times per second per CPU) reads the call stack of the running process, both kernel and user side, and aggregates it in a map. The agent periodically retrieves the aggregated stacks, symbolizes them and sends them to the server.

2. The tools #

Three open source options.

Comparison #

Parca with Docker #

docker network create prof
docker run -d --name parca --network prof -p 7070:7070 ghcr.io/parca-dev/parca:latest /parca
docker run -d --name parca-agent --network prof --privileged --pid host \
  -v /sys/kernel/debug:/sys/kernel/debug -v /lib/modules:/lib/modules:ro \
  ghcr.io/parca-dev/parca-agent:latest \
  --remote-store-address=parca:7070 --remote-store-insecure

The interface (http://localhost:7070) shows CPU profiles for all processes, filterable by labels (comm, container, pod).

Good to know: The agent's exact options change between versions: check the parca-agent README if an option is rejected.

3. Symbols #

Function names rather than addresses.

Symbolizing properly #

• Go, Rust, C/C++: keep the symbol tables (no strip, no -ldflags="-s -w") or publish the debuginfo files.
• Java, Node.js, Python: recent agents can unwind the stacks of these runtimes; check the list of runtimes supported by your agent.
• Parca can fetch symbols from a debuginfod server for distribution packages.