Refactor risk/notable querying to pin to a single savedsearch ID
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 139
- Forks
- 52
- Avg merge
- 1h 16m
- Merged PRs (30d)
- 3
Description
- In #241 I refactored risk/notable lookups to use the oldest
orig_sidinstead of the newest - The benefit of this, is that it helps ensure we don't encounter weird race conditions, where close to the end of the timeout, we are looking at the most recent run of the savedsearch and only a partial set of risk events has been generated (leads to false validation errors where an observable might not have a corresponding risk)
- More work is needed though to alleviate very occasional false successes observed as it relates to risk/observable matching
- We have validation which ensure we don't have Attacker type observables matching to risk objects (we observe this issue in about ~50 detections at the time of writing, but #234 will resolve these)
- Occasionally, a detection with one of these "bad" risk objects may pass this validation if at query time the bad risk event has not yet been generated
- This could be alleviated by instead finding a specific
orig_sid(savesearch ID) for the risk/notable in question, and pinning all future queries for risk/notables to that SID - Additionally, this would allow us to query the status of the SID job, so we might have more confidence that all risk events that are going to be generated, have been generated
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the risk/notable lookup changes from #241 and review how queries select orig_sid for each saved search. Compare the matching validation described here with the issue in #234, then determine how to retain one SID and check its job status. Done means future risk/notable queries remain pinned to that SID and account for completion of its generated events.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100