apache / apache/beam

Thread local caching of GcsIO instances does not work

Open
#18,973 0 comments 0 reactions 0 assignees View on GitHub
bug core P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

gcsio.py contains some code to cache new instances per thread:
 
```

local_state = threading.local()
if getattr(local_state, 'gcsio_instance', None) is None:

```

However, since local_state is always a newly created thread-local storage, this property will never be set.

To get the intended effect, a global threading.local() instance should be stored on the class or module level.

Imported from Jira [BEAM-5206](https://issues.apache.org/jira/browse/BEAM-5206). Original Jira may contain additional context.
Reported by: hakon.

Contributor guide

Open the contributing guide

Research direction

Start in gcsio.py by reading the thread-local caching code and checking how GcsIO instances are created. Confirm the storage for the thread-local state is shared at module or class scope, then verify that instances are reused within one thread and remain separate across threads.

Written by the indexing model from the issue text.

Assessment

Tech stack
gcp, python
Domain
cloud
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.