opensearch-project / opensearch-project/security-analytics

[BUG] Findings history deleted by alert retention instead of finding retention

Open
#1,759 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug untriaged
Dominant language
Java
Stars
111
Forks
111
Avg merge
1d 17h
Merged PRs (30d)
13

Description

Description

Summary

Findings history indices are deleted based on alert_history_retention_period, not finding_history_retention_period.

In DetectorIndexManagementService, cleanup applies alert, finding, correlation, and IOC retention checks to every scanned index. A findings index has no alerts-history alias, so once it is older than alert retention it is marked for delete—even when finding retention is longer.

Steps to reproduce
  1. Create a detector that generates findings.
  2. Apply these cluster settings:
PUT _cluster/settings
{
  "persistent": {
    "plugins.security_analytics.finding_history_max_age": "2d",
    "plugins.security_analytics.finding_history_retention_period": "4d",
    "plugins.security_analytics.alert_history_retention_period": "2d"
  }
}
  1. Generate findings and wait for the findings index to roll over after finding_history_max_age (2 days).
  2. Wait until the rolled findings index is older than alert_history_retention_period (2 days), but still younger than finding_history_retention_period (4 days).
  3. Check indices after the next cleanup job:
GET _cat/indices/.opensearch-sap-*-findings*?v
Example
  1. Day 0 — findings index created
  2. Day 2 — index rolls over (max_age)
  3. Day 2+ — old findings index is deleted because age > alert retention (2d)

Result: findings are gone after ~2 days, not 4.

Expected behavior
  • Findings indices are retained for finding_history_retention_period
  • Alert history indices are retained for alert_history_retention_period
  • Each history type uses only its own retention setting
Reference

For reference, this was the file I checked: src/main/java/org/opensearch/securityanalytics/indexmanagment/DetectorIndexManagementService.java.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/main/java/org/opensearch/securityanalytics/indexmanagment/DetectorIndexManagementService.java and trace the cleanup checks applied to scanned findings and alert history indices. Reproduce the four-day versus two-day retention scenario from the issue, then verify that each history type is deleted only according to its own retention setting.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.