epam / epam/cloud-pipeline

elasticsearch-agent doesn't delete old indexes

Open
#601 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

kind/bug priority/low sys/search
Dominant language
Java
Stars
163
Forks
65
Avg merge
11h 46m
Merged PRs (30d)
29

Description

In case of redeploy situation when old indexes aren't deleted can be possible.

the following flow describes the bug:

1. create new elastic index for storage
2. create alias for that index (now we have 2 indexes with the same alias previous and new one)
3. redeploy 
4.  create new elastic index for the same storage
5. create alias for that index (now we have 3 indexes with the same alias)
6. trying to delete old index by alias (here we delete only 1 index, so 1 old index will be stored forever)
....

by repeating this flow continuously we will get situation when a lot of old indexes stuck in the elasticsearch

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the elasticsearch-agent code that creates indexes and aliases, then trace the deletion step in the redeploy flow described in the issue. Reproduce multiple redeploys against Elasticsearch and verify that all old indexes associated with the alias are removed; no file or test is named in the issue, so add or update the relevant regression coverage you find.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, java
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.