splunk / splunk/contentctl

Refactor risk/notable querying to pin to a single savedsearch ID

Open
#248 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
139
Forks
52
Avg merge
1h 16m
Merged PRs (30d)
3

Description

  • In #241 I refactored risk/notable lookups to use the oldest orig_sid instead of the newest
  • The benefit of this, is that it helps ensure we don't encounter weird race conditions, where close to the end of the timeout, we are looking at the most recent run of the savedsearch and only a partial set of risk events has been generated (leads to false validation errors where an observable might not have a corresponding risk)
  • More work is needed though to alleviate very occasional false successes observed as it relates to risk/observable matching
    • We have validation which ensure we don't have Attacker type observables matching to risk objects (we observe this issue in about ~50 detections at the time of writing, but #234 will resolve these)
    • Occasionally, a detection with one of these "bad" risk objects may pass this validation if at query time the bad risk event has not yet been generated
  • This could be alleviated by instead finding a specific orig_sid (savesearch ID) for the risk/notable in question, and pinning all future queries for risk/notables to that SID
  • Additionally, this would allow us to query the status of the SID job, so we might have more confidence that all risk events that are going to be generated, have been generated

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the risk/notable lookup changes from #241 and review how queries select orig_sid for each saved search. Compare the matching validation described here with the issue in #234, then determine how to retain one SID and check its job status. Done means future risk/notable queries remain pinned to that SID and account for completion of its generated events.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.