elastic / elastic/elastic-package
Creating a local integration test for the dynamic template
- Dominant language
- Go
- Stars
- 72
- Forks
- 141
- Avg merge
- 19h 42m
- Merged PRs (30d)
- 55
Description
This issue is to request adding a dynamic template integration test, to make sure that the dynamic template is parsing the different ECS fields correctly, and that the correct type is set.
I have created an example document here, with an example value for all the ECS fields, including nested variations: https://gist.github.com/P1llus/4d01f857bcd4e5eaf1338ad2892f0630
The plan for the integration test is:
PS: Make sure this integration test is **NOT** blocking at the moment, because the integration test will fail with a few fields in its current state. If possible we should just have it possible to run locally, but not be included in the CI yet.
**Steps:**
1. An `elasticsearch` instance should be started.
2. An index/datastream should be created, which includes a custom setting to increase the amount of fields allowed in a specific index/datastream.
This setting is called: `index.mapping.total_fields.limit`, let's set it to something like `5000`
3. The dynamic template should be included in this index.
4. The provided example document should be POST'ed to the index.
5. Similar to a package system test, the resulting mapping/value of the field should be retrieved and checked that it corresponds with the documented ECS fields.
If possible, the test should continue even if a single mistake is found, and in the end it should return all fields that does not match the ECS definition, and what the correct type/value should be, same as with the current system tests.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the package system tests and the provided ECS example document in the linked gist. Run Elasticsearch locally, create an index or data stream with index.mapping.total_fields.limit set to about 5000, include the dynamic template, and POST the example document. Done means the resulting mappings and values are checked against the documented ECS fields without blocking CI, with all mismatches reported together.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, go
- Domain
- databases, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100