Graylog2 / Graylog2/graylog2-server
Messages returned in search results when filtering by stream even though the message does not exist in the specified streams list
- Dominant language
- Java
- Stars
- 8.1k
- Forks
- 1.1k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 217
Description
Messages returned in search results when filtering by stream even though the message does not exist in the specified streams list. Note that the message being stored more than once (in more than one index) is crucial for reproducing this bug.
This can be seen demonstrated in the below screenshot:

Note that the message is stored in 2 streams (index sets): 'second' and 'fourth'. However, when searching I have only specified messages in streams 'second' and 'third' be returned. Because stream 'third' shares a common index set with stream 'fourth', it is returning the message despite the message not existing in the stream. I've not tested using pipeline rules to route but have no reason to believe that would change this as the issue appears to be at time of search and not at time of ingestion.
More context about how the message is routed:

Context about how the search is executed:

## Expected Behavior
Graylog does not return messages that are not present in the list of streams specified in the search
## Current Behavior
Graylog is returning messages that are not present in the list of streams specified in the search.
## Possible Solution
Filter the list of log messages against the list of streams specified in the search query.
## Steps to Reproduce (for bugs)
1. Create an index set that can be used to store messages in addition to the default index set
2. Create 3 streams
* one to route messages into that uses the index set created above
* a second that will not have messages routed into it, but will use the default index set
* a third that WILL have messages routed into it, will use default index set
3. Configure stream rules for the 1st and 3rd stream created above
* This will intentionally store each message 2 times, once in index set created above, and again in the default index set
5. Search for messages and find a message that is duplicated (e.g. same gl2_message_id)
* add this to the search criteria, `gl2_message_id:`
6. Add streams to the stream filter input box:
* Add streams 1 and 2 created above. Note that we did not configure stream 2 to have any messages to route into it, but it shares the same index set as stream 3 (default index set)
7. Observe that the message appears twice despite the message existing in one of the two streams that were queried
## Context
This was reported by a customer.
## Your Environment
* Graylog Version: 5.2.2 (Reproduced this on 6.0.4)
* Java Version: Bundled
* OpenSearch Version: OpenSearch 1.3.4, OpenSearch 2.15.0
* MongoDB Version: 5.0.23, 6.0.16
* Operating System: Oracle Linux 8, Ubuntu Server 22.04
* Browser version: Version 126.0.6478.183 (Official Build) (arm64)
Please let me know if there are any questions.
Contributor guide
Assessment
This issue has not been assessed yet.