VeryGoodOpenSource / VeryGoodOpenSource/very_good_cli

feat: Support test sharding for CI parallelization with test_optimizer

Open
#1,538 1 comment 6 reactions 1 assignee View on GitHub

@ryzizub is already working on this.

Since Apr 22, 2026.

Dominant language
Dart
Stars
2.4k
Forks
244
Avg merge
1d 19h
Merged PRs (30d)
25

Description

Problem

very_good test --optimization consolidates all tests into a single .test_optimizer.dart file, which is great for reducing startup overhead. However, this single-file approach prevents CI-level sharding — splitting the test suite across multiple parallel CI runners.

For large Flutter projects, the test phase is often the CI bottleneck. In our case, PR checks take 13-23 minutes with the test phase accounting for 8-15 minutes, even with -j 4 concurrency within a single runner. Our NFR target is under 10 minutes for the full PR pipeline.

Proposal

Add --shard and --total-shards parameters (or similar) to very_good test --optimization that would:

  1. Partition the consolidated test imports across N shards deterministically (e.g., by hash or alphabetical split)
  2. Generate shard-specific optimizer files (e.g., .test_optimizer_1_of_3.dart) each containing a subset of test imports
  3. Allow CI workflows to use a strategy.matrix to run shards in parallel:
strategy:
  matrix:
    shard: [1, 2, 3]
steps:
  - run: very_good test --optimization --shard ${{ matrix.shard }} --total-shards 3

Alternatives Considered

  • -j N concurrency: Already in use, but limited to single-runner parallelism. Diminishing returns beyond core count.
  • --exclude-tags: Helps reduce scope but doesn't address the fundamental single-runner bottleneck.
  • Manual test file splitting: Defeats the purpose of test_optimizer's consolidation.
  • flutter test --shard-index: Flutter's native sharding works on individual test files, but conflicts with test_optimizer's single-file approach.

Context

This would complement the existing --optimization flag by extending its benefits to multi-runner CI setups. The test_optimizer already knows the full list of test files — partitioning them into N balanced groups seems like a natural extension.

Would also pair well with very_good_workflows if the reusable workflow added a shards input parameter.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.