Introduction #
Definition #
The score in Elasticsearch is one of the most important concepts: it defines how search results are ranked. Every time you run a search in Elasticsearch, a score is calculated for each document that matches your search criteria. The higher this value, the more relevant the document is considered to be. For each document, it is represented by the "_score" metadata field.The components #
Several factors are taken into account when calculating the score, including:1. Term frequency (TF): the number of times a term appears in a document. A higher frequency indicates a stronger relationship between the term and the document.
2. Inverse document frequency (IDF): measures how important a term is across all documents in the index. A term that appears in fewer documents has a higher IDF, which reflects its importance.
3. Field length: the length of the field in which the term appears. Shorter fields with matching terms are considered more relevant, because the term makes up a larger share of the field.
4. Boost: lets you assign a boost factor to specific fields or terms when the query runs. This can be useful to emphasise the importance of certain fields or terms in the search results.
Built-in algorithms #
Like any product, Elasticsearch evolves a great deal to add new features, fix bugs and so on. The same goes for score management. Elasticsearch supports several algorithms to calculate the score and even lets you customise them. You can go further and write your own algorithm if you have a very, very specific use case. But as always, start by testing and customising the existing algorithms before creating your own.
TD-IDF [deprecated] #
This is the algorithm that was used by default in Elasticsearch for a long time, up to version 7.
BM25 #
Also known as Okapi BM25, this is the algorithm now used by default in Elasticsearch. It is an evolution of TD-IDF and accepts the following customisation parameters when defining the similarity:1. k1: controls non-linear term frequency normalisation (saturation). Its default value is 1.2
2. b: controls to what degree document length normalises tf values. Its default value is 0.75
2. discount_overlaps: determines whether overlap tokens (tokens with a position increment of 0) are ignored when computing the norm. This option is true by default, meaning overlap tokens are not counted when computing norms.
Customising your score #
boost: at index time [deprecated] #
You can pass a boost parameter in your document. However, since version 5.0 this is discouraged, because quite often you will have to reindex all your documents whenever it changes.PUT my-index-000001
{
"mappings": {
"properties": {
"title": {
"type": "text",
"boost": 2
},
"content": {
"type": "text"
}
}
}
}boost: at search time #
You can pass a boost parameter in your search query. Add a query https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-boosting-query.html
At search time: constant_score #
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-constant-score-query.html
At search time: score function #
https://www.elastic.co/guide/en/elasticsearch/reference/8.13/index-modules-similarity.html https://opster.com/guides/elasticsearch/search-apis/elasticsearch-score/
