I think _thread._local should be treated as a GC root
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 840
- Forks
- 72
- PR merge metrics
- No merged PRs in 30d
Description
Writing this up because I'd been staring at "memory leaks" for a while that I think are actually downstream of TLS and finally (I believe) put the pieces together
From the docs:
> For simplicity’s sake let’s assume all of the roots are modules. objgraph provides a function, [is_proper_module()](https://objgraph.readthedocs.io/en/stable/objgraph.html#objgraph.is_proper_module), to check this. If you’ve any examples where that isn’t true, I’d love to hear about them (although see [Reference counting bugs](https://objgraph.readthedocs.io/en/stable/index.html#leaking-objects)).
I believe that I have another case of a GC "root" that is useful, `_thread._local`.
In the following code I dig through gc referrers for an object and it stops at `_thread._local`. `_thread._local` itself doesn't seem to have gc tracking so it doesn't link back to the module it's defined in unfortunately.
```
[0] % python test_threadlocal_theory.py
Found 1 instances of BB by by_type
DEPTH 0
{'big_boy': <__main__.BigBoy object at 0x100bc70e0>}
DEPTH 1
{<_thread._localdummy object at 0x100ab9010>: {'big_boy': <__main__.BigBoy object at 0x100bc70e0>}}
DEPTH 2
<_thread._local object at 0x100b334c0>
DEPTH 3
DEPTH 4
DEPTH 5
DEPTH 6
DEPTH 7
DEPTH 8
DEPTH 9
```
I think thread-locals will generally be dead-ends with the backref traversal algorithm, and having `objgraph.probable_gc_roots` might be useful?
```
#!/usr/bin/env python3
import gc
import threading
import objgraph
class BigBoy:
pass
tls = threading.local()
tls.big_boy = BigBoy()
bb_count = len(objgraph.by_type("BigBoy"))
print(f"Found {bb_count} instances of BB by by_type")
import threading as _threading
queue = [objgraph.by_type("BigBoy")[0]]
next_queue = []
ignore = [queue, next_queue, locals()]
for depth in range(10):
print(f"DEPTH {depth}")
while queue:
elt = queue.pop()
for ref in gc.get_referrers(elt):
if objgraph.is_proper_module(ref):
continue
if ref in ignore:
continue
print(ref)
next_queue.append(ref)
queue.extend(next_queue)
while next_queue:
next_queue.pop()
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no implementation file or test; start by running the supplied test_threadlocal_theory.py reproducer and tracing the back-reference traversal around is_proper_module(). Define the expected handling of _thread._local, then verify the chosen behavior with coverage for the reproducer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devtools
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 45/100