RousselTMLEARNING

TP · ClickStack: from discovery to expertise

Final lab: a production-ready ClickStack platform

What you will learn in this TP
  • Deploy ClickStack with Docker Compose
  • Forward file logs without loss
  • Separate write and read in ClickHouse
  • Tune retention and aggregates
  • Back up and restore
  • Validate the checklist

Introduction #

What you will learn in this section
  • Deploy ClickStack with Docker Compose
  • Forward file logs without loss
  • Separate write and read in ClickHouse
  • Tune retention and aggregates
  • Back up and restore
  • Validate the checklist

Context. The demo convinced: ClickStack must now receive the nginx front-end logs without losing any, with separate access, controlled retention and verified backups, before the go-live review.

Prerequisites: a machine with Docker (4 GB free memory), curl and jq, git  ·  Estimated time: 90 min

A ClickStack Compose stack, an outage-resistant edge collector, separate accounts, per-table retention, a restorable backup and a validated checklist.

Setup #

Guided steps, with their solution.

  1. 1. Deploy ClickStack with Compose

    Why? One container per component: each is monitored, restarted and sized separately.

    Clone the ClickStack repository, pin the image version in .env, start, create the account and export the ingestion key. Find the ClickHouse service name.
  2. 2. An edge collector for nginx

    Why? The persistent queue keeps logs if ClickStack is unavailable.

    Write edge.yaml, generate an nginx log file, then start a contrib collector on the Compose network.
  3. 3. Test outage resistance

    Why? An observability platform losing logs during an incident is useless.

    Stop the ClickStack collector, add 200 lines, restart it, then check the total reaches 700 nginx lines.
  4. 4. Separate accounts and limit readers

    Why? An analyst must neither write nor saturate the database with an unbounded query.

    Create a read-only account with a limits profile, and check it cannot drop a table.
  5. 5. Retention and HTTP code aggregate

    Why? Raw logs are expensive; their aggregate can be kept long for almost nothing.

    Set 14 days on otel_logs and 7 days on otel_traces, then create a materialized view counting nginx requests per minute and HTTP status, kept 400 days.
  6. 6. Back up ClickHouse and MongoDB

    Why? A backup never restored is only a hypothesis.

    Add a backup disk to ClickHouse (file mounted in config.d and a /backups volume via a docker-compose.override.yml), back up otel_logs, restore it under another name and compare. Back up MongoDB with mongodump.
  7. 7. Run the checklist

    Why? The go-live review asks for evidence, not impressions.

    Collect evidence: collector failed sends, parts per table, slow queries, TTL, then create in the UI a missing nginx logs alert. Note what is still missing for production (replication, TLS, single sign-on).

Difficulty level : ●●●●○ (4/5)

My progress

  • ⏱ Estimated time : ≈ 1 h 30

Sign in to track your progress and take notes. Sign in

💬 Questions about this lesson

No questions yet: be the first to ask for help or share your configuration.

Ask a question about this lesson →

Recommended Articles

Installing and deploying ClickStack: all-in-one, Docker Compose, Helm and Managed ClickStack

Deploying ClickStack to fit the need: all-in-one container with persistence, Docker Compose and i...

Sending telemetry to ClickStack: OpenTelemetry SDKs, collectors, logs, Kubernetes and browser sessions

Feeding ClickStack: ingestion key, OpenTelemetry SDK variables, forwarding through an existing co...

Operating ClickHouse for ClickStack: sizing, replication, monitoring, security and backup

Running ClickHouse in production for observability: estimating volume and disk, replicating with ...

SigNoz, ClickStack and ClickHouse best practices for observability

Best practices for observability on ClickHouse with SigNoz or ClickStack: OpenTelemetry instrumen...

Operating an observability platform on ClickHouse and choosing a tool

Operate ClickHouse for telemetry (replication, disks, merges, object storage, monitoring) and com...

ClickStack: architecture, components and offerings (ClickHouse, HyperDX, OpenTelemetry)

ClickStack, ClickHouse's official observability stack: role of ClickHouse, the UI (HyperDX), the ...

Telemetry in ClickHouse: schema, SQL, materialized views and retention

Telemetry in ClickHouse: otel_logs, otel_traces, otel_metrics_* and hyperdx_sessions tables, sort...

Observability on ClickHouse: why a columnar database changes the game

Why more and more observability platforms store logs, traces and metrics in ClickHouse: columnar ...

Exploring with the ClickStack UI: sources, search, traces, sessions and SQL

Using the ClickStack UI (HyperDX): data sources, property and full-text search syntax, SQL mode, ...

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!