googleapis / googleapis/dotnet-spanner-entity-framework
Flaky test: ConnectionTests.TestInvalidDatabase fails with Status: Unavailable
- Dominant language
- C#
- Stars
- 34
- Forks
- 13
- Avg merge
- 4h 39m
- Merged PRs (30d)
- 10
Description
### Describe the bug
The unit test `ConnectionTests.TestInvalidDatabase` is flaky and can fail on CI runs with a gRPC `Unavailable` status instead of the expected `NotFound` status.
### Error details
```
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2154838Z Failed TestInvalidDatabase [44 ms]
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2155314Z Error Message:
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2155982Z Assert.That(exception.Code, Is.EqualTo(Code.NotFound))
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2156866Z Expected: NotFound
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2157299Z But was: Unavailable
```
### Stack trace
```
at Google.Cloud.Spanner.DataProvider.Tests.ConnectionTests.TestInvalidDatabase() in /home/runner/work/dotnet-spanner-entity-framework/dotnet-spanner-entity-framework/spanner-ado-net/spanner-ado-net-tests/ConnectionTests.cs:line 249
```
### Investigation & Root Cause
The flake is caused by a race condition during the rapid teardown and restart of the Go proxy server process:
1. `TestInvalidDatabase` calls `SpannerPool.CloseSpannerLib()`, which asynchronously kills the running Go proxy server wrapper using `process.Kill()`.
2. This abruptly closes all active TCP connections to the in-memory C# mock server. The C# mock server needs a few milliseconds to clean up the closed channels.
3. The test immediately opens a new connection, which spawns a new Go proxy server.
4. The new proxy tries to connect to the C# mock server on `Fixture.Port` immediately. Since the mock server is still cleaning up from the abrupt teardown, it rejects the new connection, throwing `Unavailable`.
### Proposed Fix
Update `Server.Stop()` to wait for the process to exit synchronously after calling `Kill()`, ensuring socket and handle release is complete before execution proceeds:
```csharp
public void Stop()
{
if (_process == null || _process.HasExited)
{
return;
}
_process.Kill();
_process.WaitForExit(1000); // Wait up to 1 second for termination
}
```
Contributor guide
Research direction
Start with ConnectionTests.TestInvalidDatabase() at spanner-ado-net-tests/ConnectionTests.cs:249, then locate the Go proxy Server.Stop() implementation and inspect its process shutdown behavior. Run the focused test repeatedly, including in the CI-style environment, and confirm teardown completes before the next connection so invalid databases consistently report NotFound rather than Unavailable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100