opensearch-project / opensearch-project/security-analytics

[BUG] Error Creating Detector [security_analytics_exception] normalizer [lowercase_keyword] not found for field

Open
#697 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
111
Forks
111
Avg merge
1d 17h
Merged PRs (30d)
13

Description

What is the bug?
When trying to create a security analytics detector I get an error "[security_analytics_exception] normalizer [lowercase_keyword] not found for field 'x'". x being _ Ex. 25-000873_J6yHbIsBON-lwPZfCF7R

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Go to 'Security Analytics'
  2. Click on 'Create Detector'
  3. Configuration is for all elastic beats data sources "elastic_windows*"
  4. Log Type 'windows'
  5. Select all Rules
  6. All mappings automatically applied except for 'windows-message' which was manually mapped to 'event.original'
  7. Alert Trigger used default settings did not add channel during testing
  8. Click 'Create Detector'
  9. See error
  10. "[security_analytics_exception] normalizer [lowercase_keyword] not found for field"

What is the expected behavior?
When creating indexes we already apply normalizers like lowercase_keyword. I pasted the code we utilize in the additional context section.

What is your host/environment?

  • OS: Windows 11
  • Version: OpenSearch 2.8
  • Plugins:

Do you have any screenshots?
Screenshot 2023-10-26 110448

Do you have any additional context?
When we create an index, we added the lowercase_keyword normalizer and set it as the normalizer for the message field
{ "mappings": { "properties": { "message": { "type": "keyword", "normalizer": "lowercase_keyword" } } }, "settings": { "analysis": { "normalizer": { "lowercase_keyword": { "type": "custom", "filter": [ "lowercase" ] } } } } }

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing detector creation in Security Analytics with the described elastic_windows* source, Windows log type, mappings, and custom lowercase_keyword normalizer. Inspect how detector creation resolves field normalizers and compare it with the supplied index mapping; done when a detector using the custom-normalized field can be created without the security_analytics_exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.