googleapis / googleapis/google-cloud-go

bigquery/storage/managedwriter: resolvePool holds client mutex across GetWriteStream, serializing concurrent stream opens

Open
#20,173 0 comments 1 reaction 0 assignees View on GitHub
api: bigquery
Dominant language
Go
Stars
4.5k
Forks
1.6k
Avg merge
1d 13h
Merged PRs (30d)
109

Description

### Client

BigQuery Storage Write — `bigquery/storage/managedwriter`

### Description

`Client.resolvePool` (`bigquery/storage/managedwriter/client.go`) holds the client-wide `c.mu` across the `GetWriteStream` RPC it uses to discover a stream's region:

```go
func (c *Client) resolvePool(ctx context.Context, settings *streamSettings, streamFunc streamClientFunc) (*connectionPool, error) {
c.mu.Lock()
defer c.mu.Unlock()
resp, err := c.getWriteStream(ctx, settings.streamID, false) // network RPC, under the lock
...
}
```

That `GetWriteStream` RPC touches no shared state (only the immutable `rawClient`) — the lock exists solely to guard the `pools` map. Holding it across the RPC serializes every concurrent `NewManagedStream`/`resolvePool` on a client behind a single ~one-RTT call.

**Impact:** for workloads that open streams frequently (e.g. opening a stream per write on the `_default` stream), open latency grows linearly with concurrency. Measured cross-region (AWS us-west-2 → BigQuery US multi-region): a single uncontended open is ~115ms, but at 20 concurrent opens on one client, open latency rises to ~1s (≈ 115ms × workers) and throughput plateaus because opens cannot overlap. The RPC itself is fast; the mutex scope is the bottleneck. Present through the latest release and `main`.

**Proposed fix:** move the `GetWriteStream` call outside the lock and take `c.mu` only for the `pools` map lookup/insert. The lookup and `createPool` stay atomic under the lock, so the create-once-per-location invariant is unchanged. I have a branch + regression test ready and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

Research direction

Start in googleapis/google-cloud-go/bigquery/storage/managedwriter/client.go at Client.resolvePool and inspect the related pool lookup and creation logic. Verify the existing regression test mentioned in the issue, then confirm that concurrent stream opens no longer serialize on the client mutex while pool creation remains once per location.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, google-cloud
Domain
backend-api-design, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.