Elasticsearch intermittently killed-CodePudding

We are running Elasticsearch 8.2 on Ubuntu 20.04. Machine has 4GB of memory. Less than 1MB of data has been indexed as we are still in testing. However, Elasticsearch gets killed by the OS after a few hours.

Jun 24 01:53:27 industry-new kernel: [1443806.368450] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom
,task_memcg=/system.slice/elasticsearch.service,task=java,pid=1366207,uid=112
Jun 24 01:53:27 industry-new kernel: [1443806.368567] Out of memory: Killed process 1366207 (java) total-vm:4775580kB, anon-rss:2350732kB, f
ile-rss:0kB, shmem-rss:0kB, UID:112 pgtables:4976kB oom_score_adj:0
Jun 24 01:53:27 industry-new kernel: [1443806.495334] oom_reaper: reaped process 1366207 (java), now anon-rss:0kB, file-rss:0kB, shmem-rss:0
kB

We tried limiting the heap size by creating a file called es.options in /etc/elasticsearch/jvm.options.d and setting:

--Xms2g
--Xmx2g

With that setting in place, we could not even restart elasticsearch - it timed out.

Any suggestions on how to address these kills please?

CodePudding user response：

You need to put below lines in the jvm.options not the es.options file, and this file needs to be in the config folder of Elasticsearch.

--Xms2g
--Xmx2g

Note: Refer jvm options official document for more info.

CodePudding user response：

Was able to "solve" this by setting the 2 Xm values to 1g each. This is on a machine with 4GB of RAM, additionally running only a small python instance served by Gunicorn. The data cached in Elasticsearch does not exceed 20MB, yet if left unchecked it will eventually be killed because of memory. Even with current settings, Elastic is constantly taking up just under 2GB of RAM. Why I cannot even begin to comprehend. As I noted in my question, each machine has 4GB of memory, so one needs to set your jvm allocation at no more than 1gb to prevent oom kills.

I think we were on Elasticsearch 4 or 5 prior to the upgrade and it ran fine on a 1GB machine. Elasticsearch also deprecated the "type" variable when indexing, which makes tracking what is actually in the index unnecessarily complex. Documentation on Elastic for both Python and Ubuntu can only be described as chaotic and incomplete. I doubt we will be recommending Elasticsearch for any customer going forward.