RousselTMLEARNING

Observability

Sending telemetry to ClickStack: OpenTelemetry SDKs, collectors, logs, Kubernetes and browser sessions

Feeding ClickStack: ingestion key, OpenTelemetry SDK variables, forwarding through an existing collector, file logs, Kubernetes, browser sessions with the HyperDX SDK, Map or JSON schema, throughput and security.

1. Endpoints and ingestion key #

Everything goes through OTLP.

Principle #

Available in: ✓ Open source ✓ Managed ClickStack

ClickStack receives all telemetry through its OpenTelemetry collector: OTLP HTTP on 4318 or gRPC on 4317. Each request must carry the ingestion key, shown in the UI (Team Settings › API Keys), in the authorization header.

Watch out : The ingestion key lets anyone write into your database: store it as a secret, and do not expose OTLP ports to the Internet without TLS or filtering (except for the browser SDK, behind a dedicated endpoint).

2. Instrumenting applications #

OpenTelemetry SDKs and agents.

Standard variables #

export OTEL_EXPORTER_OTLP_ENDPOINT=http://clickstack:4318
export OTEL_EXPORTER_OTLP_HEADERS="authorization=$CLICKSTACK_INGESTION_KEY"   # Team Settings › API Keys
export OTEL_SERVICE_NAME=checkout
export OTEL_RESOURCE_ATTRIBUTES=deployment.environment.name=production,service.version=2.4.1
# Java example: OpenTelemetry agent, no code change
java -javaagent:opentelemetry-javaagent.jar -jar checkout.jar

These variables apply to every language (Java, .NET, Node.js, Python, Go…). ClickStack also provides SDKs wrapping OpenTelemetry that accept the key in their init function.

Attributes to get right #

  • service.name: the ServiceName column, main UI filter.
  • deployment.environment.name, service.version: separate production and staging, spot a regression.
  • Logs emitted through the OpenTelemetry SDK (or correlated): they carry TraceId and SpanId, which lets you go from a trace to its logs.

3. Forwarding through a collector #

File logs, enrichment, buffering.

Configuration #

A collector close to the applications (host agent or Kubernetes DaemonSet) reads log files, enriches data and forwards it to ClickStack with the ingestion key.

# Existing collector (contrib) forwarding to ClickStack: file logs and application OTLP
receivers:
  otlp:
    protocols: { grpc: { endpoint: 0.0.0.0:4317 }, http: { endpoint: 0.0.0.0:4318 } }
  filelog:
    include: [/var/log/nginx/*.log]
    operators:
      - type: regex_parser
        regex: '^(?P<client>\S+) \S+ \S+ \[(?P<time>[^\]]+)\] "(?P<method>\S+) (?P<path>\S+) \S+" (?P<status>\d{3})'
processors:
  batch: { send_batch_size: 10000, timeout: 5s }
  memory_limiter: { check_interval: 1s, limit_percentage: 80, spike_limit_percentage: 20 }
  resourcedetection: { detectors: [env, system] }
exporters:
  otlphttp/clickstack:
    endpoint: http://clickstack:4318
    headers: { authorization: ${env:CLICKSTACK_INGESTION_KEY} }
    compression: gzip
service:
  pipelines:
    logs:   { receivers: [otlp, filelog], processors: [memory_limiter, resourcedetection, batch], exporters: [otlphttp/clickstack] }
    traces: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [otlphttp/clickstack] }
    metrics: { receivers: [otlp], processors: [memory_limiter, batch], exporters: [otlphttp/clickstack] }
  telemetry:
    metrics: { address: 0.0.0.0:9888 }    # two collectors on one host: change the internal metrics port

Kubernetes #

  • A DaemonSet collector reads pod logs (/var/log/pods) and node metrics (kubeletstats); the k8sattributes processor adds namespace, pod and deployment.
  • A Deployment collector gathers cluster metrics (k8s_cluster) and events, once for the whole cluster.
  • The official OpenTelemetry Collector Helm chart provides both modes (kubernetesAttributes, logsCollection, kubeletMetrics presets).

4. Browser sessions #

Replay linked to traces.

The browser SDK #

// npm install @hyperdx/browser: sessions, errors and traces in the browser
import HyperDX from '@hyperdx/browser';

HyperDX.init({
  url: 'https://otel.example.com',                 // collector OTLP HTTP endpoint (exposed over HTTPS)
  apiKey: '<ingestion-key>',
  service: 'shop-web',
  tracePropagationTargets: [/api\.exemple\.fr/i],  // propagates traceparent to the back end
  consoleCapture: true,
  advancedNetworkCapture: false,                 // only enable request body capture knowingly
});
HyperDX.setGlobalAttributes({ userId: 'u-1842' }); // opaque ID, never an e-mail

Sessions are stored in the hyperdx_sessions table. In the UI, a back-end error leads to the trace, then to the browser session that triggered it, with its replay.

Best practice : Replay masks input fields by default; check that no personal data appears in replayed pages and collect consent if needed.

5. Schema, throughput and security #

The settings that matter.

Map or JSON? #

By default, attributes are stored in Map(LowCardinality(String), String) columns (LogAttributes, SpanAttributes, ResourceAttributes): it is the recommended choice. A schema using ClickHouse's JSON type exists in beta (enabled by a collector feature gate, clickhouse.json): keep it for trials with a small, stable set of attributes.

Throughput #

  • ClickHouse prefers few inserts of large batches: keep the batch processor (thousands of events, a few seconds).
  • Add memory_limiter and, for spikes, a persistent queue (file_storage extension) so nothing is lost during an outage.
  • Filter out what has no value at the source (health probes, debug logs): every stored line costs disk.

Commands to know #

Commands to know
  • OTEL_EXPORTER_OTLP_ENDPOINT=http://clickstack:4318 : collector address
  • OTEL_EXPORTER_OTLP_HEADERS=authorization=<clé> : ingestion key
  • curl -s localhost:8888/metrics | grep otelcol_exporter_sent : events sent by the collector
  • curl -s localhost:8888/metrics | grep send_failed : failed sends
  • SELECT ServiceName, count() FROM otel_logs WHERE Timestamp > now() - INTERVAL 5 MINUTE GROUP BY ServiceName : check logs are arriving

Sources #

Checked in November 2026: ClickStack evolves fast (Managed ClickStack is in beta), check the ClickHouse documentation before relying on a detail.

Related articles

The observability trilemma: cost, completeness and performance, and how solutions resolve it

Understanding the observability trilemma (cost, data completeness, query performance), its three trade-offs, …

Installing and deploying ClickStack: all-in-one, Docker Compose, Helm and Managed ClickStack

Deploying ClickStack to fit the need: all-in-one container with persistence, Docker Compose and its variables…

Exploring with the ClickStack UI: sources, search, traces, sessions and SQL

Using the ClickStack UI (HyperDX): data sources, property and full-text search syntax, SQL mode, live tail, t…

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!