opensearch-project / opensearch-project/security-analytics
[BUG] Findings history deleted by alert retention instead of finding retention
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 111
- Forks
- 111
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 13
Description
Description
Summary
Findings history indices are deleted based on alert_history_retention_period, not finding_history_retention_period.
In DetectorIndexManagementService, cleanup applies alert, finding, correlation, and IOC retention checks to every scanned index. A findings index has no alerts-history alias, so once it is older than alert retention it is marked for delete—even when finding retention is longer.
Steps to reproduce
- Create a detector that generates findings.
- Apply these cluster settings:
PUT _cluster/settings
{
"persistent": {
"plugins.security_analytics.finding_history_max_age": "2d",
"plugins.security_analytics.finding_history_retention_period": "4d",
"plugins.security_analytics.alert_history_retention_period": "2d"
}
}
- Generate findings and wait for the findings index to roll over after
finding_history_max_age(2 days). - Wait until the rolled findings index is older than
alert_history_retention_period(2 days), but still younger thanfinding_history_retention_period(4 days). - Check indices after the next cleanup job:
GET _cat/indices/.opensearch-sap-*-findings*?v
Example
- Day 0 — findings index created
- Day 2 — index rolls over (
max_age) - Day 2+ — old findings index is deleted because age > alert retention (2d)
Result: findings are gone after ~2 days, not 4.
Expected behavior
- Findings indices are retained for
finding_history_retention_period - Alert history indices are retained for
alert_history_retention_period - Each history type uses only its own retention setting
Reference
For reference, this was the file I checked: src/main/java/org/opensearch/securityanalytics/indexmanagment/DetectorIndexManagementService.java.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/main/java/org/opensearch/securityanalytics/indexmanagment/DetectorIndexManagementService.java and trace the cleanup checks applied to scanned findings and alert history indices. Reproduce the four-day versus two-day retention scenario from the issue, then verify that each history type is deleted only according to its own retention setting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, security
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100