opensearch-project / opensearch-project/security-analytics

[BUG] Rule query index template misses log types with empty mappings (e.g. apache_access)

Open
#1,640 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
111
Forks
111
Avg merge
1d 17h
Merged PRs (30d)
13

Description

What is the bug?
Description
RuleTopicIndices builds the index_patterns for the rule query index template using LogTypeService.getAllLogTypes().
This method is based on a terms aggregation over FieldMappingDoc.log_types, so log types with empty mappings are excluded.

Example: apache_access is a built‑in log type, but its OSMapping/apache_access_logtype.json has an empty mappings array, so it does not appear in the log_types aggregation and is therefore missing from .opensearch-sap-detectors-queries-index-template.

As a result, when a rule query index is created for apache_access, the settings from detector-settings.json (including rule_analyzer) are not applied, and monitor creation fails when it tries to PUT the mapping. The error log below shows this:

[2026-02-02T07:06:45,659][ERROR][o.o.a.u.DocLevelMonitorQueries] [manager1] unknown exception during PUT mapping on queryIndex: .opensearch-sap-apache_access-detectors-queries-optimized-1ef37621-6d48-40a5-b50a-3ddf59086b79-000001, retrying with deletion of query indexjava.lang.IllegalArgumentException: analyzer [rule_analyzer] has not been configured in mappings    at org.opensearch.index.mapper.ParametrizedFieldMapper$Parameter.lambda$analyzerParam$0(ParametrizedFieldMapper.java:521) ~[opensearch-3.4.0.jar:3.4.0]    ...

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Create a detector of type apache_access (dedicated query indices enabled or disabled).
  2. Inspect the generated rule query index (e.g. .opensearch-sap-apache_access-detectors-queries*).
  3. Check whether the index settings/mappings include rule_analyzer.

What is the expected behavior?

All built‑in log types (all log types present in metadata) should have their rule query indices created with the common settings from .opensearch-sap-detectors-queries-index-template, including rule_analyzer.

What is your host/environment?

  • OS: docker
  • Version [e.g. 22]
  • Plugins

Do you have any screenshots?
If applicable, add screenshots to help explain your problem.

Do you have any additional context?
Add any other context about the problem.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at RuleTopicIndices and LogTypeService.getAllLogTypes(), then inspect how the rule query index template is built from log-type metadata. Reproduce with the built-in apache_access type and check whether the generated template includes rule_analyzer. Done means all built-in log types, including those with empty mappings, receive the common query-index settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.