elastic / elastic/elastic-package

[Enhancement Request] Enabling testing of non-cloud packages with the latest software version.

Open
#1,902 12 comments 0 reactions 0 assignees View on GitHub
stale
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

Many of the integration package system tests do not use the product's latest (defined in `variants.yml`) docker image for testing.

The integration packages must be tested against all the latest software versions to remain compatible.

To tackle this, the below mentioned approach is expected

- Developers, as part of the local system run (`elastic-package system test`) are intimated there exist a major version of the product available.
- Developers are expected to do one of the following approach
- Update `variants.yml` with the information of the latest (major version) of the software package. If the test is successful, create a PR to permanently include the major version of the software in the `variants.yml`
- Add entry in the `validations.yml` to suppress the notification.

**Current Environment.**

- Elastic-package is the most used tool by an integration developer as part of package development and testing.
- Elastic-package has capabilities to validate the quality of the integration package. If a specific package does not meet the required capabilities, exclusions can be added to validation.yml file of the specific package
- Elastic-package tool is aware of the folder structure of the integration package through package-spec

**Requirement**

- Include (non-cloud) software version check as part of the Integration system test

Sample
```
DOCKERFILE=Dockerfile
IMAGE_LINE=$(grep -E '^FROM' $DOCKERFILE)
IMAGE_NAME=$(echo $IMAGE_LINE | awk -F':' '{print $1}' | awk '{print $2}')
IMAGE_TAG=$(echo $IMAGE_LINE | awk -F':' '{print $2}')

echo "Image name: $IMAGE_NAME"
echo "Current version: $IMAGE_TAG"

LATEST_VERSION=$(curl -s "https://registry.hub.docker.com/v2/repositories/library/$IMAGE_NAME/tags?page_size=100" | jq -r '.results[].name' | grep -E '^[0-9]+\.[0-9]+\.[0-9]+$' | sort -V | tail -n1)

echo "Latest version: $LATEST_VERSION"

echo "::set-output name=image-name::$IMAGE_NAME"
echo "::set-output name=current-version::$IMAGE_TAG"
echo "::set-output name=latest-version::$LATEST_VERSION"

```

- Add a new validation, with a validation code such that - if the package is not using the latest version of the software, an error is reported. This error can be ignored by adding a validation exclusion in the validation.yml file
- Record the test outcome as an artifact file.

```
--report-format json --report-output file
--report-format xUnit --report-output file
```

**Note:**
The existing validation settings in elastic-package make use of elastic's package-spec. Since these settings are only related to test (system test) , making use of package-spec validation approach may not be needed i guess. However, making use of a common (`validations.yml`) file may be helpful

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.