googleapis / googleapis/dotnet-spanner-entity-framework

Flaky test: ConnectionTests.TestInvalidDatabase fails with Status: Unavailable

Open Beginner friendly
#775 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
34
Forks
13
Avg merge
4h 39m
Merged PRs (30d)
10

Description

### Describe the bug
The unit test `ConnectionTests.TestInvalidDatabase` is flaky and can fail on CI runs with a gRPC `Unavailable` status instead of the expected `NotFound` status.

### Error details
```
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2154838Z Failed TestInvalidDatabase [44 ms]
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2155314Z Error Message:
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2155982Z Assert.That(exception.Code, Is.EqualTo(Code.NotFound))
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2156866Z Expected: NotFound
units (9.0.x, ubuntu-latest) spanner-ado-net-tests 2026-06-24T11:52:28.2157299Z But was: Unavailable
```

### Stack trace
```
at Google.Cloud.Spanner.DataProvider.Tests.ConnectionTests.TestInvalidDatabase() in /home/runner/work/dotnet-spanner-entity-framework/dotnet-spanner-entity-framework/spanner-ado-net/spanner-ado-net-tests/ConnectionTests.cs:line 249
```

### Investigation & Root Cause
The flake is caused by a race condition during the rapid teardown and restart of the Go proxy server process:
1. `TestInvalidDatabase` calls `SpannerPool.CloseSpannerLib()`, which asynchronously kills the running Go proxy server wrapper using `process.Kill()`.
2. This abruptly closes all active TCP connections to the in-memory C# mock server. The C# mock server needs a few milliseconds to clean up the closed channels.
3. The test immediately opens a new connection, which spawns a new Go proxy server.
4. The new proxy tries to connect to the C# mock server on `Fixture.Port` immediately. Since the mock server is still cleaning up from the abrupt teardown, it rejects the new connection, throwing `Unavailable`.

### Proposed Fix
Update `Server.Stop()` to wait for the process to exit synchronously after calling `Kill()`, ensuring socket and handle release is complete before execution proceeds:
```csharp
public void Stop()
{
if (_process == null || _process.HasExited)
{
return;
}
_process.Kill();
_process.WaitForExit(1000); // Wait up to 1 second for termination
}
```

Contributor guide

Open the contributing guide

Research direction

Start with ConnectionTests.TestInvalidDatabase() at spanner-ado-net-tests/ConnectionTests.cs:249, then locate the Go proxy Server.Stop() implementation and inspect its process shutdown behavior. Run the focused test repeatedly, including in the CI-style environment, and confirm teardown completes before the next connection so invalid databases consistently report NotFound rather than Unavailable.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.