RousselTMLEARNING

Article · ClickStack: from discovery to expertise

Alerts and dashboards

1. Charts #

Aggregating any field.

Building a chart #

Available in: ✓ Open source ✓ Managed ClickStack

A chart starts from a source and a search, then applies an aggregation (count, sum, mean, min, max, percentiles, distinct values) on a field, optionally grouped by another (service, route, HTTP status).

ChartSource and searchAggregation
Errors per serviceLogs · level:errcount, grouped by ServiceName
p95 latency per routeTraces · service:checkout95th percentile of Duration, grouped by http.route
Active usersSessionsdistinct user IDs
HTTP codesLogs · web servicecount, grouped by http.status_code

2. Dashboards #

Gathering useful charts.

Best practices #

  • One dashboard per service or journey, built around RED signals: request rate, errors, duration.
  • Global filters (environment, service) applied to every chart on the dashboard.
  • Search tables (latest error logs) next to curves, to jump straight to details.
  • Dashboards are stored in MongoDB: back it up, or export the definition of important dashboards.

What about Grafana? #

Grafana's ClickHouse plugin queries the same tables in SQL: useful if the team already lives in Grafana or wants to mix ClickHouse with Prometheus. Both UIs can coexist on the same data.

-- Grafana panel (ClickHouse plugin): errors per service, per minute
SELECT $__timeInterval(Timestamp) AS time, ServiceName, count() AS errors
FROM otel_logs
WHERE $__timeFilter(Timestamp) AND SeverityText ILIKE 'err%'
GROUP BY time, ServiceName ORDER BY time

3. Alerts #

Being warned, not flooded.

Two types #

Alert on…Example
a saved searchmore than 50 service:checkout level:err logs in 5 minutes
a chartcheckout p95 latency above 800 ms over 10 minutes
missing datano payment service logs for 10 minutes (“below 1” threshold)

Each alert has an evaluation interval, a threshold (above or below) and destinations: Slack, e-mail, generic webhook (to Grafana IRM, PagerDuty, Opsgenie…). The message contains a link to the search or chart, hence the importance of HYPERDX_APP_URL.

Making them reliable #

  • Alert on user-visible symptoms (errors, latency) rather than on every error log.
  • Choose a window long enough to ignore a single spike (5 to 10 minutes) and a threshold based on history.
  • Add a missing-data alert on critical services: a stopped collector triggers no error alert.
  • Test each alert by triggering the condition in staging, and check the message link.
Watch out : Alerts depend on the UI (evaluation) and MongoDB (definitions): when self-hosted, monitor both, otherwise alerts silently stop.

Commands to know #

Commands to know
  • HYPERDX_APP_URL : address used in alert links
  • level:err service:checkout : search to turn into an alert
  • $__timeFilter(Timestamp) : Grafana ClickHouse plugin time filter
  • mongodump --db hyperdx : back up dashboards and alerts (self-hosted)

Sources #

Checked in November 2026: ClickStack evolves fast (Managed ClickStack is in beta), check the ClickHouse documentation before relying on a detail.

📄 Alerts and dashboards in ClickStack →

My progress

  • ⏱ Estimated time : ≈ 5 min

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

SigNoz, ClickStack and ClickHouse best practices for observability

Best practices for observability on ClickHouse with SigNoz or ClickStack: OpenTelemetry instrumen...

Exploring with the ClickStack UI: sources, search, traces, sessions and SQL

Using the ClickStack UI (HyperDX): data sources, property and full-text search syntax, SQL mode, ...

Operating an observability platform on ClickHouse and choosing a tool

Operate ClickHouse for telemetry (replication, disks, merges, object storage, monitoring) and com...

Using SigNoz: APM, traces, logs, dashboards and alerts

Use SigNoz day to day: APM view of services, traces and logs explorers, query builder, ClickHouse...

Observability on ClickHouse: why a columnar database changes the game

Why more and more observability platforms store logs, traces and metrics in ClickHouse: columnar ...

ClickStack: architecture, components and offerings (ClickHouse, HyperDX, OpenTelemetry)

ClickStack, ClickHouse's official observability stack: role of ClickHouse, the UI (HyperDX), the ...

Alerts, connectors and SLOs in Kibana

Create alert rules in Kibana (Elasticsearch query, ES|QL, custom threshold, logs), configure conn...

Operating ClickHouse for ClickStack: sizing, replication, monitoring, security and backup

Running ClickHouse in production for observability: estimating volume and disk, replicating with ...

The observability trilemma: cost, completeness and performance, and how solutions resolve it

Understanding the observability trilemma (cost, data completeness, query performance), its three ...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!