VeryGoodOpenSource / VeryGoodOpenSource/very_good_test_runner

fix: `flutter` command gets locked when trying to run sharded tests parallely

Open
#25 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug p2
Dominant language
Dart
Stars
20
Forks
2
Avg merge
1d 7h
Merged PRs (30d)
1

Description

Description
When running multiple flutter test commands parallely via flutterTest() method for sharded tests, the cli throws the below error

Waiting for another flutter command to release the startup lock...

If the same flutter test command is ran for sharded tests without using flutterTest() method but via Process.start, the flutter command runs the sharded tests successfully.

Steps To Reproduce

  1. Call the below methods simultaneously
flutterTest(
          arguments:
          "--total-shards 2 --shard-index 0 '${testDirectory.path}/example_a[1]_test.dart'"
              .split(' '),
          workingDirectory: tempDirectory.path,
        );

flutterTest(
          arguments:
          "--total-shards 2 --shard-index 1 '${testDirectory.path}/example_a[2]_test.dart'"
              .split(' '),
          workingDirectory: tempDirectory.path,
        );

Expected Behavior
When executing the flutter test command via Process.start for the shared tests, it runs without failure.
flutterTest() method also uses Process.start internally & it should run without locking flutter command on the sharded tests.

Process.start(
    'flutter',
    "--total-shards 2 --shard-index 1 '${testDirectory.path}/example_a[1]_test.dart'"
              .split(' '),
  );
Process.start(
    'flutter',
    "--total-shards 2 --shard-index 2 '${testDirectory.path}/example_a[2]_test.dart'"
              .split(' '),
  );

Screenshots
image

Additional Context
Please find the test case to reproduce the issue here:

test('emits correctly for sharded tests (e2e)', () async {
      final tempDirectory = Directory.systemTemp.createTempSync();
      File('${tempDirectory.path}/pubspec.yaml').writeAsStringSync(
        '''
name: example
version: 0.1.0+1

environment:
  sdk: ">=2.12.0 <3.0.0"

dev_dependencies:
  test: any
''',
      );
      final testDirectory = Directory('${tempDirectory.path}/test')
        ..createSync();
      File('${testDirectory.path}/example_a[1]_test.dart').writeAsStringSync(
        '''
import 'package:test/test.dart';

void main() {
  test('example', () {
    expect(true, isTrue);
  });
}
''',
      );

      File('${testDirectory.path}/example_a[2]_test.dart').writeAsStringSync(
        '''
import 'package:test/test.dart';

void main() {
  test('example', () {
    expect(true, isTrue);
  });
}
''',
      );
      expect(
        flutterTest(
          arguments:
              "--total-shards 2 --shard-index 0 '${testDirectory.path}/example_a[1]_test.dart'"
                  .split(' '),
          workingDirectory: tempDirectory.path,
        ).where((e) => e is DoneTestEvent).first,
        completes,
      );

      expect(
        flutterTest(
          arguments:
          "--total-shards 2 --shard-index 1 '${testDirectory.path}/example_a[2]_test.dart'"
              .split(' '),
          workingDirectory: tempDirectory.path,
        ).where((e) => e is DoneTestEvent).first,
        completes,
      );
    });

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the flutterTest() implementation and compare its process handling with the direct Process.start examples. Reproduce the issue using the provided sharded-test setup and inspect why simultaneous commands wait on Flutter's startup lock. Done means both flutterTest() calls complete successfully without the lock error.

Written by the indexing model from the issue text.

Assessment

Tech stack
dart, flutter
Domain
cli, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.