elastic / elastic/elastic-package

Validate variables specified in system tests

Open
#2,186 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

When a developer writes a system test config like [_dev/test/system/test-*-config.yml](https://github.com/elastic/integrations/blob/8882cd426ac9c7ac34d8a0b60c9e793777e89256/packages/1password/data_stream/audit_events/_dev/test/system/test-default-config.yml), they can specify variables that are to be passed in the `POST /api/fleet/package_policies` fleet API request.

As was observed in elastic/integrations#11554, there is no validation that the variables specified are correct. The feature request is to make elastic-package validate that the variables used in the test config actually correspond to real variables defined in the package or data stream manifests, and that they match the "level" at which they are defined in the package.

The fact that incorrect variables are accepted probably means the POST /api/fleet/package_policies fleet API is not strict about validating the specified parameters exist.

Surprisingly, there are only two levels of variables listed in the [code](https://github.com/elastic/elastic-package/blob/700dd69a9df381475fbdc3440d01db093ec233f9/internal/testrunner/runners/policy/testconfig.go#L20-L23) when there are four levels of variables in packages (integration, policy template, policy template input, data stream). We should probably expand the test config to match with 4 "levels".

Related

- https://github.com/elastic/integrations/pull/11554

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.