elastic / elastic/elastic-package

Creating a local integration test for the dynamic template

Open
#1,096 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

This issue is to request adding a dynamic template integration test, to make sure that the dynamic template is parsing the different ECS fields correctly, and that the correct type is set.

I have created an example document here, with an example value for all the ECS fields, including nested variations: https://gist.github.com/P1llus/4d01f857bcd4e5eaf1338ad2892f0630

The plan for the integration test is:
PS: Make sure this integration test is **NOT** blocking at the moment, because the integration test will fail with a few fields in its current state. If possible we should just have it possible to run locally, but not be included in the CI yet.

**Steps:**
1. An `elasticsearch` instance should be started.
2. An index/datastream should be created, which includes a custom setting to increase the amount of fields allowed in a specific index/datastream.
This setting is called: `index.mapping.total_fields.limit`, let's set it to something like `5000`
3. The dynamic template should be included in this index.
4. The provided example document should be POST'ed to the index.
5. Similar to a package system test, the resulting mapping/value of the field should be retrieved and checked that it corresponds with the documented ECS fields.

If possible, the test should continue even if a single mistake is found, and in the end it should return all fields that does not match the ECS definition, and what the correct type/value should be, same as with the current system tests.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the package system tests and the provided ECS example document in the linked gist. Run Elasticsearch locally, create an index or data stream with index.mapping.total_fields.limit set to about 5000, include the dynamic template, and POST the example document. Done means the resulting mappings and values are checked against the documented ECS fields without blocking CI, with all mismatches reported together.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, go
Domain
databases, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.