1. Charts #
Aggregating any field.
Building a chart #
Available in: ✓ Open source ✓ Managed ClickStack
A chart starts from a source and a search, then applies an aggregation (count, sum, mean, min, max, percentiles, distinct values) on a field, optionally grouped by another (service, route, HTTP status).
| Chart | Source and search | Aggregation |
|---|---|---|
| Errors per service | Logs · level:err | count, grouped by ServiceName |
| p95 latency per route | Traces · service:checkout | 95th percentile of Duration, grouped by http.route |
| Active users | Sessions | distinct user IDs |
| HTTP codes | Logs · web service | count, grouped by http.status_code |
2. Dashboards #
Gathering useful charts.
Best practices #
- One dashboard per service or journey, built around RED signals: request rate, errors, duration.
- Global filters (environment, service) applied to every chart on the dashboard.
- Search tables (latest error logs) next to curves, to jump straight to details.
- Dashboards are stored in MongoDB: back it up, or export the definition of important dashboards.
What about Grafana? #
Grafana's ClickHouse plugin queries the same tables in SQL: useful if the team already lives in Grafana or wants to mix ClickHouse with Prometheus. Both UIs can coexist on the same data.
-- Grafana panel (ClickHouse plugin): errors per service, per minute
SELECT $__timeInterval(Timestamp) AS time, ServiceName, count() AS errors
FROM otel_logs
WHERE $__timeFilter(Timestamp) AND SeverityText ILIKE 'err%'
GROUP BY time, ServiceName ORDER BY time3. Alerts #
Being warned, not flooded.
Two types #
| Alert on… | Example |
|---|---|
| a saved search | more than 50 service:checkout level:err logs in 5 minutes |
| a chart | checkout p95 latency above 800 ms over 10 minutes |
| missing data | no payment service logs for 10 minutes (“below 1” threshold) |
Each alert has an evaluation interval, a threshold (above or below) and destinations: Slack, e-mail, generic webhook (to Grafana IRM, PagerDuty, Opsgenie…). The message contains a link to the search or chart, hence the importance of HYPERDX_APP_URL.
Making them reliable #
- Alert on user-visible symptoms (errors, latency) rather than on every error log.
- Choose a window long enough to ignore a single spike (5 to 10 minutes) and a threshold based on history.
- Add a missing-data alert on critical services: a stopped collector triggers no error alert.
- Test each alert by triggering the condition in staging, and check the message link.
Watch out : Alerts depend on the UI (evaluation) and MongoDB (definitions): when self-hosted, monitor both, otherwise alerts silently stop.
Commands to know #
Commands to know
HYPERDX_APP_URL: address used in alert linkslevel:err service:checkout: search to turn into an alert$__timeFilter(Timestamp): Grafana ClickHouse plugin time filtermongodump --db hyperdx: back up dashboards and alerts (self-hosted)
Sources #
Checked in November 2026: ClickStack evolves fast (Managed ClickStack is in beta), check the ClickHouse documentation before relying on a detail.
My progress
Sign in to track your progress and take notes. Sign in
💬 Questions about this lesson
No questions yet: be the first to ask for help or share your configuration.
Recommended Articles
SigNoz, ClickStack and ClickHouse best practices for observability
Best practices for observability on ClickHouse with SigNoz or ClickStack: OpenTelemetry instrumen...
Exploring with the ClickStack UI: sources, search, traces, sessions and SQL
Using the ClickStack UI (HyperDX): data sources, property and full-text search syntax, SQL mode, ...
Operating an observability platform on ClickHouse and choosing a tool
Operate ClickHouse for telemetry (replication, disks, merges, object storage, monitoring) and com...
Using SigNoz: APM, traces, logs, dashboards and alerts
Use SigNoz day to day: APM view of services, traces and logs explorers, query builder, ClickHouse...
Observability on ClickHouse: why a columnar database changes the game
Why more and more observability platforms store logs, traces and metrics in ClickHouse: columnar ...
ClickStack: architecture, components and offerings (ClickHouse, HyperDX, OpenTelemetry)
ClickStack, ClickHouse's official observability stack: role of ClickHouse, the UI (HyperDX), the ...
Alerts, connectors and SLOs in Kibana
Create alert rules in Kibana (Elasticsearch query, ES|QL, custom threshold, logs), configure conn...
Operating ClickHouse for ClickStack: sizing, replication, monitoring, security and backup
Running ClickHouse in production for observability: estimating volume and disk, replicating with ...
The observability trilemma: cost, completeness and performance, and how solutions resolve it
Understanding the observability trilemma (cost, data completeness, query performance), its three ...