1. Charts #
Aggregating any field.
Building a chart #
Available in: ✓ Open source ✓ Managed ClickStack
A chart starts from a source and a search, then applies an aggregation (count, sum, mean, min, max, percentiles, distinct values) on a field, optionally grouped by another (service, route, HTTP status).
| Chart | Source and search | Aggregation |
|---|---|---|
| Errors per service | Logs · level:err | count, grouped by ServiceName |
| p95 latency per route | Traces · service:checkout | 95th percentile of Duration, grouped by http.route |
| Active users | Sessions | distinct user IDs |
| HTTP codes | Logs · web service | count, grouped by http.status_code |
2. Dashboards #
Gathering useful charts.
Best practices #
- One dashboard per service or journey, built around RED signals: request rate, errors, duration.
- Global filters (environment, service) applied to every chart on the dashboard.
- Search tables (latest error logs) next to curves, to jump straight to details.
- Dashboards are stored in MongoDB: back it up, or export the definition of important dashboards.
What about Grafana? #
Grafana's ClickHouse plugin queries the same tables in SQL: useful if the team already lives in Grafana or wants to mix ClickHouse with Prometheus. Both UIs can coexist on the same data.
-- Grafana panel (ClickHouse plugin): errors per service, per minute
SELECT $__timeInterval(Timestamp) AS time, ServiceName, count() AS errors
FROM otel_logs
WHERE $__timeFilter(Timestamp) AND SeverityText ILIKE 'err%'
GROUP BY time, ServiceName ORDER BY time3. Alerts #
Being warned, not flooded.
Two types #
| Alert on… | Example |
|---|---|
| a saved search | more than 50 service:checkout level:err logs in 5 minutes |
| a chart | checkout p95 latency above 800 ms over 10 minutes |
| missing data | no payment service logs for 10 minutes (“below 1” threshold) |
Each alert has an evaluation interval, a threshold (above or below) and destinations: Slack, e-mail, generic webhook (to Grafana IRM, PagerDuty, Opsgenie…). The message contains a link to the search or chart, hence the importance of HYPERDX_APP_URL.
Making them reliable #
- Alert on user-visible symptoms (errors, latency) rather than on every error log.
- Choose a window long enough to ignore a single spike (5 to 10 minutes) and a threshold based on history.
- Add a missing-data alert on critical services: a stopped collector triggers no error alert.
- Test each alert by triggering the condition in staging, and check the message link.
Watch out : Alerts depend on the UI (evaluation) and MongoDB (definitions): when self-hosted, monitor both, otherwise alerts silently stop.
Commands to know #
Commands to know
HYPERDX_APP_URL: address used in alert linkslevel:err service:checkout: search to turn into an alert$__timeFilter(Timestamp): Grafana ClickHouse plugin time filtermongodump --db hyperdx: back up dashboards and alerts (self-hosted)
Sources #
Checked in November 2026: ClickStack evolves fast (Managed ClickStack is in beta), check the ClickHouse documentation before relying on a detail.
