engineyard / engineyard/ey-cloud-recipes

Elastic Search should terminate & restart gracefully

Open
#91 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
977
Forks
283
PR merge metrics
No merged PRs in 30d

Description

From one of our customers John Paul:

> ```
> check process elasticsearch_9080
> with pidfile /var/run/elasticsearch.pid
> start program = "/usr/lib/elasticsearch/bin/elasticsearch -p /var/run/elasticsearch.pid" with timeout 90 seconds
> stop program = "/bin/bash -c '/bin/kill `cat /var/run/elasticsearch.pid`'" with timeout 90 seconds
> group elasticsearch
> ```
>
> kill is just program sends a TERM signal to a process by default. it does not guarantee the shutdown process, so after a Kill, Monit thinks that ES is off. but > in reality it is not so!
>
> Next, monit starts a new process elasticsearch. Together with the old process, they form a cluster. New shard begins to process and download the data from the old shard, but do not have time, as the old process shutdowned by kill.
>
> This can cause vanishing indexes
>
> This situation can be avoided by using standard upstart or systemd as supervisor of demons.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.