googleapis / googleapis/google-cloud-python
Bigquery Python Library: `AppendRows` bidi-streaming RPC sends empty `x-goog-request-params` routing header, causing intermittent `InvalidArgument: Cannot route on empty project id ''``
- Ngôn ngữ chính
- Python
- Star
- 5.4k
- Fork
- 1.8k
- Merge trung bình
- 3 ngày 4 giờ
- Pull request đã merge (30 ngày)
- 122
Mô tả
### Determine this is the right repository
- [x] I determined this is the correct repository in which to report this bug.
### Summary of the issue
**Context**
Using `BigQueryWriteClient.append_rows()` to write data via the Storage Write API default stream (`_default`), with multiple sequential calls through the same client. This is in a multiprocessing setup where each worker creates its own `BigQueryWriteClient()` and makes many `append_rows` calls.
**Expected Behavior:**
`append_rows` should populate the `x-goog-request-params` gRPC routing header with the `write_stream` resource name (which contains the project ID), similar to how other methods in the same client do it:
```python
# create_write_stream — sets routing correctly
metadata = ... + (gapic_v1.routing_header.to_grpc_metadata((("parent", request.parent),)),)
```
**Actual Behavior:**
`append_rows` sends an empty routing header because it's a bidi-streaming RPC and the request iterator hasn't been consumed when metadata is set:
```python
# client.py, append_rows() — all versions 2.26.0 through 2.37.0
metadata = tuple(metadata) + (gapic_v1.routing_header.to_grpc_metadata(()),)
```
This produces `x-goog-request-params: ''`. The first several calls may succeed via the gateway's fallback routing, but after a gRPC reconnection (idle timeout, load balancing), the gateway cannot determine the target project and returns:
```
google.api_core.exceptions.InvalidArgument: 400 Cannot route on empty project id ''
```
**Reproduction:**
```python
from google.cloud.bigquery_storage_v1 import BigQueryWriteClient
from google.api_core.gapic_v1 import routing_header
import inspect
# Verify empty routing in source
source = inspect.getsource(BigQueryWriteClient.append_rows)
assert "to_grpc_metadata(())" in source
# Verify it produces an empty header
assert routing_header.to_grpc_metadata(()) == ("x-goog-request-params", "")
```
**Workaround:**
Pass routing metadata explicitly:
```python
metadata = (routing_header.to_grpc_metadata((("write_stream", stream_name),)),)
response = client.append_rows(requests=iter([request]), metadata=metadata)
```
**Environment:**
- `google-cloud-bigquery-storage`: 2.26.0 through 2.37.0 (all affected)
- Python 3.9
### API client name and version
_No response_
### Reproduction steps: code
file: main.py
```python
def reproduce():
# complete code here
```
### Reproduction steps: supporting files
file: mydata.csv
```
alpha,1,3
beta,2,5
```
### Reproduction steps: actual results
file: output.txtmydata.csv
```
Calculated: foo
```
### Reproduction steps: expected results
file: output.txtmydata.csv
```
Calculated: bar
```
### OS & version + platform
_No response_
### Python environment
_No response_
### Python dependencies
_No response_
### Additional context
_No response_
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.