splunk / splunk/contentctl

Bug: Contentctl adds spaces in annotations when outputting to savedsearches.conf in ESCU

Open
#142 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
139
Forks
52
Avg merge
1h 16m
Merged PRs (30d)
3

Description

When creating a correlation search manually via the GUI in Splunk Enterprise Security, the annotations are created without spaces. An example:

{"cis20":["CIS 10"],"kill_chain_phases":["Exploitation"],"mitre_attack":["T1003.001","T1003"],"nist":["DE.CM"],"confidence":90,"impact":90,"analytic_story":["Credential Dumping"]}

The same query done via ESCU has the following annotations:
{"analytic_story": ["Credential Dumping"], "cis20": ["CIS 10"], "confidence": 90, "impact": 90, "kill_chain_phases": ["Exploitation"], "mitre_attack": ["T1003.001", "T1003"], "nist": ["DE.CM"], "type": "TTP"}

After every 'annotation name', there is a space after the colon. No one probably even caught this, but I was working on rex to extract the analytic stories, mitre_attack techniques, confidence, and impact from the annotations which causes the problem to appear.

My initial code, with ESCU only correlation searches worked just fine:

| rex field=action.correlationsearch.annotations "\"analytic_story\": \[(?<analytic_story>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"mitre_attack\": \[(?<mitre_attack>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"confidence\": (?<confidence>[^,]+)," 
| rex field=action.correlationsearch.annotations "\"impact\": (?<impact>[^,]+),"

And to get it to work with custom created content I merely had to modify it to:

| rex field=action.correlationsearch.annotations "\"analytic_story\":\s?\[(?<analytic_story>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"mitre_attack\":\s?\[(?<mitre_attack>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"confidence\":\s?(?<confidence>[^,]+)," 
| rex field=action.correlationsearch.annotations "\"impact\":\s?(?<impact>[^,]+),"

Ultimately not a huge deal on my part, after realizing it. That said, I imagine contentctl's point is to make the savedsearches.conf to be in the same standards with Splunk Enterprise Security, so I'm pointing this bug out.

I am including a screenshot of proof. In it there are three results :
ESCU - Create Remote Thread into LSASS - Rule is the correlation search by default from ESCU.
BSDE - Create Remote Thread into LSASS - Rule is the same correlation search cloned via the ES GUI.
Manual Creation - Create Remote Thread into LSASS is the same correlation search, but done via the New Correlation search menu, with the data copy/pasted into it.

This can also be recreated by creating any manual correlation search with annotations to find the lack of space.

This discovery was done in collaboration with @rivosyke

contentctlbug

The code block from my screenshot is provided, so you don't have to handjam to recreate it if needed:

| rest splunk_server=local count=0 /servicesNS/-/-/saved/searches 
| search action.correlationsearch.label=* 
| rex field=action.correlationsearch.annotations "\"analytic_story\":\s?\[(?<analytic_story>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"mitre_attack\":\s?\[(?<mitre_attack>[^\]]+)\]" 
| rex field=action.correlationsearch.annotations "\"confidence\":\s?(?<confidence>[^,]+)," 
| rex field=action.correlationsearch.annotations "\"impact\":\s?(?<impact>[^,]+),"
| table action.correlationsearch.label  eai:acl.app analytic_story confidence impact mitre_attack action.correlationsearch.annotations

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the annotation output in savedsearches.conf and compare an ESCU-created correlation search with one created through the Splunk Enterprise Security GUI. Trace the contentctl entry point that generates savedsearches.conf; done means its annotation formatting matches the GUI output without spaces after JSON colons.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.