RousselTMLEARNING

Observability

Elastic Agent: architecture, Fleet and integrations

Understand Elastic Agent: single collection agent, Fleet and Fleet Server, agent policies, integrations, data streams, outputs, and managed versus standalone mode.

1. What is Elastic Agent? #

One agent for all collection.

The principle #

Elastic Agent is Elastic's single agent for collecting logs, metrics, security data and OpenTelemetry telemetry on a host, container or cluster. It replaces installing several Beats (Filebeat, Metricbeat, Winlogbeat…) with one process, configured by a policy listing the integrations to run.

Agents send data to Elasticsearch and receive their policy from Fleet Server.
Two separate flows: data goes to Elasticsearch, control goes through Fleet Server.
Good to know: Elastic Agent is free and part of Elastic's default distribution; some integrations (Elastic Defend, advanced security connectors) depend on your subscription.

What runs in an agent #

The agent is a supervisor: it receives the policy, starts the required components (derived from Beats, Endpoint for Elastic Defend, Osquery, OpenTelemetry Collector), monitors their health and handles upgrades.

Supervisor, inputs, specialised components and output.

2. Fleet, Fleet Server and policies #

Managing thousands of agents from Kibana.

The roles #

ComponentRole
FleetKibana app: policies, integrations, enrollment, upgrades, agent status
Fleet ServerComponent (a special Elastic Agent) that distributes policies and collects agent status; port 8220 by default
Agent policySet of integrations and settings applied to a group of agents (e.g. « prod web servers »)
IntegrationVersioned package: collection config, ingest pipelines, mappings, Kibana dashboards
Elastic Package RegistryIntegration catalogue (public or self-hosted for air-gapped sites)

Managed or standalone mode #

Fleet-managedStandalone
ConfigurationPolicy in Kibanaelastic-agent.yml file
UpgradesFrom Fleet, in batchesWith your tools (Ansible, image)
PrerequisitesFleet ServerNone: direct shipping to Elasticsearch
Typical useServer fleets, many teamsGitOps, ephemeral containers, isolated sites
Good practice: Start in managed mode to discover integrations, then export the policy as YAML if you want to version it in standalone mode.

3. Where does data go? #

Data streams and outputs.

Data streams #

Each integration writes to data streams named type-dataset-namespace: logs-nginx.access-default, metrics-system.cpu-prod. The namespace, chosen in the policy, separates environments or teams and drives access rights.

GET _data_stream/logs-nginx.access-*
GET logs-nginx.access-default/_search?size=1

Outputs #

  • Elasticsearch: default, direct.
  • Logstash: heavy transformations, routing, buffering.
  • Kafka: decoupling and large-scale buffering.
  • Remote Elasticsearch: ship to a cluster other than Fleet's.
Common mistake: Mixing up the Fleet Server URL (control, port 8220) and the Elasticsearch URL (data, port 9200 or 443): the agent enrolls but sends no data.

Related articles

Elastic Agent or Beats? Choosing and migrating

Compare Elastic Agent and Beats (Filebeat, Metricbeat, Winlogbeat), know when to keep Beats, and migrate step…

Installing Fleet Server and enrolling Elastic Agents

Set up Fleet Server, create a policy, enroll Linux, Windows and Docker agents, understand tokens and certific…

Agent policies, integrations, data streams and pipelines

Build agent policies, add integrations (System, Nginx, Custom Logs), choose namespaces, customise ingestion w…

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!