elastic / elastic/elastic-package
[system tests] Ignored fields are not detected in stacks 8.15.0 onward
- Dominant language
- Go
- Stars
- 72
- Forks
- 141
- Avg merge
- 19h 42m
- Merged PRs (30d)
- 55
Description
The current search performed to get the ignored fields in a given data stream is not working successfully for Elastic stack versions: 8.15.0
https://github.com/elastic/elastic-package/blob/091c6636bb00aa2acbbb2c2120c76c65eb6392e3/internal/testrunner/runners/system/tester.go#L46-L78
For Elastic stack >= 8.15.0, that `my_ignored` runtime field is not populated and it is not present in the search response, even for the data streams that contain some ignored field.
Example: this build from windows package (integrations repo) fails with 8.14.0, but it finishes successfully when it runs with 8.15.0 or 8.17.0
https://buildkite.com/elastic/integrations/builds/22752
To be tested if this query works for >=8.15.0
```json
{
"fields": ["*"],
"aggs": {
"all_ignored": {
"filter": {
"exists": {
"field": "_ignored"
}
},
"aggs": {
"ignored_fields": {
"terms": {
"size": 100,
"field": "_ignored"
}
},
"ignored_docs": {
"top_hits": {
"size": 5
}
}
}
}
}
}
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in internal/testrunner/runners/system/tester.go at lines 46-78 and reproduce the ignored-field search against Elastic stack versions 8.15.0 and later. Test the proposed aggregation query and compare its response with the current search, using the Windows package build as a reference. Done means system tests detect ignored fields consistently on the affected versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, go
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100