Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

Resolving Common Elasticsearch Errors and Exceptions

Tech Sep 4 1

This article outlines several typical errors encountered in Elasticsearch and provides their solutions.


1. Disk Full Leads to Read-Only Index

When the disk becomes full, Elasticsearch may set indices to read-only. After expanding the disk, the following errer may persist:

blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]

To resolve, set all indices to non-read-only:

curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

Refer to the official documentation for more details.


2. index.highlight.max_analyzed_offset Limit Ecxeeded

Increase the maximum analyzed offset for highlighting:

PUT my-index-2023.02.03/_settings
{
  "index" : {
    "highlight.max_analyzed_offset" : 6000000
  }
}

3. Maximum Number of Shards Reached

Error:

{
  "error": {
    "root_cause": [{
      "type": "validation_exception",
      "reason": "Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"
    }],
    "type": "validation_exception",
    "reason": "Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1000]/[1000] maximum shards open;"
  },
  "status": 400
}

Solutions: There are three ways to increase the maximum shards per node.

  • Using Kibana Console
PUT /_cluster/settings
{
  "persistent": {
    "cluster": {
      "max_shards_per_node": 10000
    }
  }
}
  • Using the Command Line
curl -XPUT http://localhost:9200/_cluster/settings -H 'Content-Type:application/json' -d '
{
  "persistent": {
    "cluster": {
      "max_shards_per_node": 10000
    }
  }
}'
  • Modifying the Configuration File
# elasticsearch.yml
cluster.max_shards_per_node: 10000

4. Request Entity Too Large (ContentTooLongException)

Error: org.apache.http.ContentTooLongException: entity content is too long [111214807] for the configured buffer limit [104857600]

Adjust the buffer limit when using the Java High Level REST Client:

RequestOptions customOptions = RequestOptions.DEFAULT.toBuilder()
    .setHttpAsyncResponseConsumerFactory(
        new HttpAsyncResponseConsumerFactory.HeapBufferedResponseConsumerFactory(500 * 1024 * 1024) // 500 MB
    )
    .build();

SearchRequest searchReq = new SearchRequest(indexName);
searchClient.search(searchReq, customOptions);

SearchScrollRequest scrollReq = new SearchScrollRequest(scrollId);
searchClient.scroll(scrollReq, customOptions);

5. Immense Term in Field Exceeds Max Length

Error: Document contains at least one immense term in field="content" (whose UTF8 encoding is longer than the max length 32766)...

For keyword fields, the default maximum length ignored is 32766. Set ignore_above to skip longer terms:

PUT /articles/_mapping
{
  "properties": {
    "writer": {
      "type": "keyword",
      "ignore_above": 32766
    }
  }
}

6. Circuit Breaking Exception: Data Too Large

Error:

{
  "error" : {
    "root_cause" : [{
      "type" : "circuit_breaking_exception",
      "reason" : "[parent] Data too large, data for [<http_request>] would be [1010545440/963.7mb], which is larger than the limit of [986061209/940.3mb]...",
      "bytes_wanted" : 1010545440,
      "bytes_limit" : 986061209
    }],
    "type" : "circuit_breaking_exception",
    "reason" : "[parent] Data too large, data for [<http_request>] would be [1010545440/963.7mb]...",
    "bytes_wanted" : 1010545440,
    "bytes_limit" : 986061209
  },
  "status" : 429
}

To fix, increase the heap size by editing config/jvm.options:

-Xms10g
-Xmx10g

These are common issues with straightforward resolutions. Always ensure your cluster configurations align with your data and query requirements.

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.