flutter / flutter/devtools

Clean up integration test sharding

Open
#6,386 0 comments 1 reaction 0 assignees View on GitHub
infra integration tests P3 tech debt
Dominant language
Dart
Stars
1.7k
Forks
404
Avg merge
6d 17h
Merged PRs (30d)
18

Description

Right now, we have an ugly matrix to implement the sharding: https://github.com/flutter/devtools/pull/6385

To improve this we could pull out the integration test into it's own callable workflow that we pass a shard number to:

Calling the 3 devices would then look something like this;
```
  flutter-integration-test:
    name: integration-test ${{ matrix.bot }} - flutter
    needs: flutter-prep
    runs-on: macos-latest

    uses: ./github/workflows/integration-test.yaml
      with:
        shards: 3
        device: flutter
 
  flutter-web-integration-test:
    name: integration-test ${{ matrix.bot }} - flutter-web
    needs: flutter-prep
    runs-on: macos-latest

    uses: ./github/workflows/integration-test.yaml
      with:
        shards: 3
        device: flutter-web
```

All of the steps and environment variables would then be pulled into the ./github/workflows/integration-test.yaml
workflow file using a callable workflow dispatch https://docs.github.com/en/actions/using-workflows/reusing-workflows#calling-a-reusable-workflow

Then to handle the sharding weirdness we could use seperate step at the top of that workflow to generate and array that would be fed to the matrix:
See the "From Json to Fully Dynamic Matrix" section here https://tomasvotruba.com/blog/2020/11/16/how-to-make-dynamic-matrix-in-github-actions

the step to generate the array would essentially make it so we could have:matrix:
`shard: ${{}}`

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing integration-test sharding matrix and PR 6385, then read the proposed ./github/workflows/integration-test.yaml structure and GitHub's reusable-workflow documentation. Confirm how the shard array should be generated and passed into the matrix. Done means the device workflows call the reusable workflow and retain equivalent integration-test coverage with simpler sharding configuration.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, yaml
Domain
ci-cd
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.