googleapis / googleapis/google-cloud-python

Synchronous Firestore call unbinds the caller thread's `contextvars` Context (breaks Flask request context) under gunicorn sync worker

未关闭
#17,821 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
api: firestore priority: p3 type: question
主要语言
Python
星标
5.4k
派生
1.8k
平均合并
3 天 4 小时
30 天内合并 PR
122

描述

### Determine this is the right repository

- [x] I determined this is the correct repository in which to report this bug.

### Summary of the issue

A **synchronous** Firestore call (gRPC transport) made from inside a request-handling
thread switches/unbinds that thread's `contextvars` Context partway through. Any
framework that stores per-request state in a `ContextVar` (Flask 2.3+ keeps its request
context in `_cv_request`) sees that state disappear on the *same* thread immediately
after the Firestore call. In our Flask app this surfaces as:

RuntimeError: Working outside of request context

raised during response finalization — a 500 to the client, even though the view code
completed successfully.

Firebase support directed us here, and pointed to a similar regression in the gRPC repo:
**grpc/grpc#40123** ("contextvar loss when using thread pool executors in newer gRPC
versions"). Filing so the Cloud Firestore team can determine whether this is global gRPC
behavior or specific to the Firestore client's transport.

## Environment

- Python: 3.9
- google-cloud-firestore: 2.18.0
- firebase-admin: 6.5.0 (thin wrapper over the client above)
- google-api-core: 2.19.2
- grpcio / grpcio-status: 1.66.1
- Transport: default (gRPC)
- Web stack: Flask 2.3.0 / Werkzeug 3.0.3, served by gunicorn **sync** worker
(one request at a time per worker process; stable OS thread)
- Client init: module-level `firestore.client()` (channel created lazily on first RPC)

## Observed behavior

We instrumented `contextvars.copy_context` and logged `has_request_context()` (hrc)
with a full stack at each call. Within a **single** request, on the **same** OS thread:

**(A) First Firestore call — hrc is True (context healthy). Fires at channel creation:**

```
grpc/_channel.py:1761 _run_channel_spin_thread -> cygrpc.ForkManagedThread(target=channel_spin)
grpc/_channel.py:1821 create -> _run_channel_spin_thread(state)
grpc/_channel.py:1414 __call__ -> self._managed_call(...)
google/api_core/grpc_helpers.py:164 error_remapped_callable
google/api_core/timeout.py / retry_unary.py / gapic_v1/method.py
google/cloud/firestore_v1/services/firestore/client.py:1154 batch_get_documents
google/cloud/firestore_v1/document.py:402 get() # <- our doc_ref.get()
... application view ... gunicorn sync worker.handle_request
[hrc=True]
```

**(B) A later Firestore call in the SAME request — hrc is now False (context gone):**

```
grpc/_channel.py:1146 _blocking -> self._channel.segregated_call(...)
grpc/_channel.py:1178 __call__ -> self._blocking(...)
google/api_core/grpc_helpers.py:76 error_remapped_callable
google/api_core/timeout.py / retry_unary.py / gapic_v1/method.py
google/cloud/firestore_v1/services/firestore/client.py:1372 commit
google/cloud/firestore_v1/batch.py:59 commit() # <- our batch.commit()
... same application/gunicorn stack ...
[hrc=False]
```

Between (A) and (B) our code runs only in-process work (SQLAlchemy queries + a list
comprehension) — nothing that touches `contextvars`, spawns threads, or uses
asyncio/greenlets. The only machinery that manipulates context between the two points
is the Firestore/gRPC transport (`_run_channel_spin_thread` / `cygrpc.ForkManagedThread`
at channel creation, and `segregated_call` on subsequent RPCs).

## Expected vs actual

- **Expected:** a synchronous Firestore RPC leaves the *calling* thread's `contextvars`
Context unchanged.
- **Actual:** after the RPC, the calling thread is running in a different Context; any
`ContextVar` set before the call (e.g. Flask's `_cv_request`) reads as unset.

## Reproduction

Flask 2.3 app served by gunicorn **sync** worker. In a view: log `has_request_context()`,
do a synchronous Firestore read+write (`doc_ref.get()` then `batch.commit()`), log
`has_request_context()` again. On the first Firestore-touching request after a worker
starts (fresh gRPC channel), the second check is False and Flask raises "Working outside
of request context" during response finalization. Minimal standalone repro available on
request.

## Questions

1. Is the Firestore Python client (or the underlying gRPC transport) expected to alter
the *calling* thread's `contextvars` Context during a synchronous RPC?
2. Is this the same root cause as grpc/grpc#40123, or specific to the Firestore client's
`ForkManagedThread` / `segregated_call` usage?
3. Supported ways to avoid it:
- Pre-creating/warming the gRPC channel at process (post-fork) startup — does that
help, or does `segregated_call` disturb context on every RPC regardless of channel
age?
- Using the REST transport (`transport="rest"`) — is that a supported path to avoid
the gRPC thread/context machinery entirely?
- A known-good `grpcio` version to pin below the regression, if this is version-specific?

## Current workaround

We run all Firestore calls on a short-lived daemon thread so the context churn lands on
a disposable thread and the request thread's context stays intact. This resolves the
500s, but we would prefer a supported fix rather than thread-isolating every call.

## Impact

Intermittent 500s (correlated with the first Firestore call per worker / gRPC channel
creation) on any endpoint that makes a synchronous Firestore call, in production.

贡献指南

打开贡献指南

调研方向

Start by reproducing the synchronous get()/batch.commit() flow under Flask 2.3 and gunicorn sync workers, then inspect grpc/_channel.py around _run_channel_spin_thread, ForkManagedThread, and segregated_call. Compare the behavior with grpc/grpc#40123 and the Firestore client call sites in google/cloud/firestore_v1; done means identifying whether the context change is gRPC-wide or Firestore-specific and documenting a supported fix or workaround.

由索引模型根据 Issue 内容生成。

评估

技术栈
flask, grpc, python
领域
backend, databases
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
冷清
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。