RousselTMLEARNING

Observabilité

OpenSearch: concepts, architecture and differences with Elasticsearch

Understanding OpenSearch: project origin, license, differences with Elasticsearch, indexes, shards, replicas, cluster manager, cluster states and components.

1. OpenSearch at a glance #

An open source search and analytics engine.

Overview #

OpenSearch is a distributed search and analytics engine built on Apache Lucene, with its console OpenSearch Dashboards. It serves application search, log analytics, observability, security and, increasingly, vector search. All its components are published under the Apache 2.0 license.

Its history with Elasticsearch #

In 2021, Elastic moved Elasticsearch and Kibana away from the Apache 2.0 license. AWS and contributors then created OpenSearch from the last Apache 2.0 versions (7.10). Since 2024, the project has been hosted by the OpenSearch Software Foundation, within the Linux Foundation. Elastic in turn added an AGPL option to its licenses in 2024.

PointOpenSearchElasticsearch
LicenseApache 2.0Elastic License, SSPL or AGPL, at your choice
Security, alerting, ISMIncluded pluginsDepends on the subscription
Index lifecycleISM (_plugins/_ism)ILM
LanguagesDSL, SQL, PPLDSL, SQL, ES|QL
ConsoleOpenSearch DashboardsKibana
The two projects have diverged. Concepts and much of the API remain common, but do not assume that a configuration, client or plugin is compatible: check. Some official Elasticsearch clients refuse to connect to OpenSearch; use the OpenSearch clients.

2. Concepts #

Indexes, shards and replicas.

Vocabulary #

ConceptMeaning
DocumentA JSON object
IndexA collection of documents
ShardA piece of an index, which is a Lucene index
Primary / replicaThe original and its copy
MappingThe type of each field
Cluster managerNode managing cluster state: the former “master”

The cluster #

An OpenSearch cluster with its shards and replicas
The number of primaries is set at index creation; the number of replicas can change at any time.
StateMeaningWhat to do?
greenAll shards are assignedNothing
yellowSome replicas are unassignedNormal with a single node; otherwise, add a node
redSome primaries are missingUrgent: data is unavailable

3. Components #

What surrounds the engine.

The ecosystem #

ComponentPurpose
OpenSearch DashboardsExploration, visualizations, administration
Data PrepperCollection and transformation, recommended for ingestion
Included pluginsSecurity, Alerting, Index Management, SQL/PPL, k-NN, Observability, Anomaly Detection…

The official 3.9 archive ships these plugins and its own JDK; list them with GET _cat/plugins.

Related articles

Installing OpenSearch and OpenSearch Dashboards

Install OpenSearch with Docker Compose or the archive: initial password, vm.max_map_count, demo certificates …

Indexing and searching in OpenSearch: mapping, DSL, PPL and SQL

Declare an index template, index with bulk, search with the Query DSL, aggregate and query with PPL and SQL. …

Feeding OpenSearch: ingest pipelines and Data Prepper

Transform data with ingest pipelines, collect with Data Prepper, receive OTLP and avoid the dynamic mapping p…

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!