grpc / grpc/grpc

[Python] Regression when using contextvar with thread pool executor

Open
#40,123 3 comments 1 reaction 1 assignee Assigned to @asheshvidyut View on GitHub
kind/bug lang/Python priority/P2
Dominant language
C++
Stars
45.3k
Forks
11.4k
Avg merge
2d 12h
Merged PRs (30d)
47

Description

### What version of gRPC and what language are you using?
Tested with: 1.59.5, 1.72.1, 1.73.1

### What operating system (Linux, Windows,...) and version?
MacOSX 15.5, Amazon Linux 2023

### What runtime / compiler are you using (e.g. python version or version of gcc)
Python 3.11

### What did you do?
We are using Django framework inside grpc calls and with newer (than 1.59.5) we are loosing value contextvar inside thread pool worker. Django framework uses contextvar to store the memcached connection (using asgiref). https://github.com/django/django/blob/main/django/core/cache/__init__.py#L39

We have a server with ThreadPoolExecutor with max_workers=5.
And inside server we call cache.get that is using contextvar to store the memcache connection.

However contextvar for each request using the same thread is lost:
```
save_id = contextvars.ContextVar[str]('save_id', default="missing")

class Greeter(helloworld_pb2_grpc.GreeterServicer):
def SayHello(self, request, context):
print(f"Id1: {save_id.get()}")
save_id.set("BLABLA")
print(f"Id2: {save_id.get()}")
return helloworld_pb2.HelloReply(message="Hello, %s!" % request.name)
```

### What did you expect to see?
GRPC 1.59.1
```
Id1: missing
Id2: BLABLA
ThreadId: 14042755072

Id1: BLABLA
Id2: BLABLA
ThreadId: 14042755072
```

### What did you see instead?
when upgrading to latest GRPC 1.72.1 and 1.73.1
```
Id1: missing
Id2: BLABLA
ThreadId: 13928656896

Id1: missing
Id2: BLABLA
ThreadId: 13928656896
```

### Anything else we should know about your project / environment?
Implementation of connection general connection pooling:
https://github.com/django/django/blob/main/django/utils/connection.py
https://github.com/django/django/blob/main/django/core/cache/__init__.py#L39

Two posibilities that Django is using:
- thread_critical=False -> memcache, redis
- thread_critical=True -> mysql, postgresql

Basically when using threadlocal everything is working. When using contextvar the value is lost.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.