hashicorp / hashicorp/terraform-exec

testing: Consider using CI version matrix instead of hard-coding latest versions

Open
#309 2 comments 0 reactions 0 assignees View on GitHub
ci testing thinking
Dominant language
Go
Stars
786
Forks
129
PR merge metrics
No merged PRs in 30d

Description

Instead of maintaining a list of version numbers in [internal/testutil/tfcache.go](https://github.com/hashicorp/terraform-exec/blob/cb3a405d1356be591b81e07ac394ac5171b685ac/tfexec/internal/testutil/tfcache.go#L14), use a version matrix in the CircleCI config. The `TFEXEC_E2ETEST_VERSIONS` env var should then be used for similar behaviour during local development.

Note that if we remove these consts entirely, tests not using the main `runTest()` helper to run tests against the default list of versions will need to hard-code the relevant versions. However, such tests often make use of latest versions, e.g.:
https://github.com/hashicorp/terraform-exec/blob/cb3a405d1356be591b81e07ac394ac5171b685ac/tfexec/internal/e2etest/upgrade012_test.go#L14-L16

Perhaps the consts should instead be initialised according to values in the CircleCI environment.

Contributor guide

Open the contributing guide

Research direction

Start with internal/testutil/tfcache.go and the CircleCI configuration, then inspect runTest() and upgrade012_test.go for tests that depend on the default version list. Determine how the CI version matrix and TFEXEC_E2ETEST_VERSIONS should provide versions in CI and local development, while preserving behavior for tests that do not use runTest().

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
ci-cd, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.