cloudflare / cloudflare/wrangler-action

Refactor tests to be integration tests rather than E2E deployments

Open
#194 0 comments 1 reaction 0 assignees View on GitHub
maintenance triaged
Dominant language
TypeScript
Stars
2k
Forks
214
PR merge metrics
No merged PRs in 30d

Description

The way this action is currently being tested is primarily through a GitHub Actions [workflow](https://github.com/cloudflare/wrangler-action/blob/0e6f0d30805b05d6dbd8861f317900d045526d80/.github/workflows/deploy.yml) that deploys several Worker fixtures and passes as long as the action doesn't fail to complete.

This approach has a few severe drawbacks. Because deploying these Workers to our test account requires our API token, PRs submitted by external contributors don't have access to our secrets and these always fail to run. These failures also mask those that might potentially occur during other checks like code formatting. This results in us merging sometimes-broken code due to having no reliable confidence signal before merging.

Second, the current E2E structure makes it difficult to assert against expected behaviors of the action in an automated way. It's also cumbersome to construct tests for each scenario (and tear them down after the test completes, etc.) These E2E tests end up mostly testing the underlying behavior of wrangler rather than the behavior of the action itself.

We can improve our test coverage by moving towards integration testing for much of the behavior currently being tested (to varying degrees) in an E2E manner. To do this, we'll invoke the action's entrypoint in a vitest and assert that the expected subprocesses are spawned appropriately.

We can limit E2E tests to a few basic smoke tests that only run `if: ${{ github.repository_owner == 'cloudflare' }}` while allowing the rest of the test suite to run on external PRs.

Contributor guide

Open the contributing guide

Research direction

Start with .github/workflows/deploy.yml and the action's entrypoint, then review how the current E2E deployments and Worker fixtures are invoked. Use Vitest to exercise the entrypoint and assert expected subprocesses, retaining only a few smoke tests gated to the cloudflare repository owner; done means external PRs can run reliable integration coverage without deployment secrets.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
ci-cd, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.