1. The data model #
A log enriched with context.
LogRecord #
OpenTelemetry does not create a new logging framework: it defines a common data model and bridges to existing libraries. A log record contains:
| Field | Content |
|---|---|
| Timestamp / ObservedTimestamp | time of the event / time of its collection |
| SeverityNumber / SeverityText | normalized 1-24 (9 = INFO, 13 = WARN, 17 = ERROR) / original text |
| Body | the message (text or structure) |
| Attributes | context (semconv: code.function.name, exception.type…) |
| TraceId / SpanId / TraceFlags | the trace and span active when the log was written |
| Resource | the emitting service, identical to its traces and metrics |
| EventName | name of a structured event (« event » logs) |
Why it is useful #
- A single format, whatever the language or logging library.
- Automatic correlation: each log carries the identifier of the current trace.
- The same resource as the other signals: filter logs, traces and metrics of the same pod in one click.
- The same transport (OTLP) and the same Collector as the other signals.
2. From the application #
Appenders and the Logs Bridge API.
The principle #
The Logs Bridge API is not meant to be called by your code: it is for authors of appenders (or handlers) that plug Logback, Log4j, java.util.logging, Python's logging module, Serilog or Winston into the SDK. You keep writing logger.info(...).
Examples #
# Python : avec l'auto-instrumentation
export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
export OTEL_LOGS_EXPORTER=otlp
opentelemetry-instrument python app.py
# Java : l'agent capture Logback et Log4j par défaut
java -javaagent:opentelemetry-javaagent.jar -Dotel.logs.exporter=otlp -jar app.jar# Python : configuration manuelle du handler
import logging
from opentelemetry._logs import set_logger_provider
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.http._log_exporter import OTLPLogExporter
provider = LoggerProvider(resource=resource)
provider.add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter()))
set_logger_provider(provider)
logging.getLogger().addHandler(LoggingHandler(level=logging.INFO, logger_provider=provider))Good practice: Even without exporting logs over OTLP, addtrace_idandspan_idto your JSON logs: correlation will also work with file-based collection.
3. From files #
The Collector's filelog receiver.
Parsing JSON logs #
receivers:
filelog:
include: [/var/log/app/*.log]
start_at: end
storage: file_storage # reprendre là où on s'était arrêté
operators:
- type: json_parser
timestamp: { parse_from: attributes.time, layout: "%Y-%m-%dT%H:%M:%S%z" }
severity: { parse_from: attributes.level }
- type: trace_parser
trace_id: { parse_from: attributes.trace_id }
span_id: { parse_from: attributes.span_id }
- type: move
from: attributes.message
to: bodyKubernetes logs #
On Kubernetes, a Collector deployed as a DaemonSet reads /var/log/pods/*/*/*.log, uses the container operator to decode the runtime format, then the k8sattributes processor to add pod, namespace, deployment and labels. The official Helm chart enables this with the logsCollection preset.
Common mistake: Do not collect the same logs twice (OTLP appender and file): doubled volume and duplicates in the backend. Choose one path per application.
4. Processing and querying #
Reduce, mask, find.
In the Collector #
processors:
filter/debug:
logs:
log_record: ['severity_number < SEVERITY_NUMBER_INFO'] # supprimer DEBUG et TRACE
transform/pii:
log_statements:
- context: log
statements:
- replace_pattern(body, "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+", "***@***")In Loki #
{service_name="checkout"} | severity_text="ERROR"
{service_name="checkout"} | trace_id="4bf92f3577b34da6a3ce929d0e0e4736"
sum by (service_name) (count_over_time({deployment_environment_name="production"} | severity_text="ERROR" [5m]))My progress
Sign in to track your progress and take notes. Sign in
💬 Recent discussions about this course
- Quels attributs utiliser pour nommer mes spans HTTP (semconv) ?✓ solved · 1 reply(ies)
- OpenTelemetry Collector : mes traces arrivent mais pas mes métriques✓ solved · 2 reply(ies)
Ask a question about this lesson →See all discussions about this course
Recommended Articles
OpenTelemetry traces in depth: spans, status, events, links and sampling
Master the trace signal: anatomy of a distributed trace, span and SpanKind, attributes, status an...
Correlating traces, metrics, logs and profiles with OpenTelemetry
Link the four signals to diagnose quickly: shared resource, trace_id in logs, exemplars on metric...
Profiles, OpenTelemetry's fourth signal: continuous profiling and correlation with traces
Understand OpenTelemetry's new profile signal: continuous profiling, flame graphs, data model and...
OpenTelemetry: automatic vs manual instrumentation
Understand the differences between automatic and manual instrumentation with OpenTelemetry, their...
Datadog APM and OpenTelemetry: traces, services and correlation with logs
Instrument applications for Datadog APM (dd-trace, single step instrumentation, OpenTelemetry), u...
Capstone project: the four signals of the OpenTelemetry Demo in the Grafana stack
A capstone project spanning the observability courses: deploy the OpenTelemetry Demo, send metric...
Ingesting OpenTelemetry logs into Elasticsearch
Collect and structure logs with the OpenTelemetry Collector (filelog receiver, operators, transfo...
Linking traces and profiles: span profiles with Tempo and Pyroscope
Go from a slow trace to the exact CPU profile of that span: span profiles in Go and Java, configu...
Sending traces to Tempo: OpenTelemetry instrumentation and Alloy
Produce and route traces to Tempo: Python/Java/Node auto-instrumentation, the manual SDK, OTEL en...