Torn reads of GC statistics from external process via get_gc_stats
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
If an external tool sampling GC statistics does not pause the target process (and we designed get_gc_stats not to pause), it can read torn data under certain circumstances.
There are two distinct problems:
-
Weakly ordered platforms: The stores of
ts_startandts_stopmay be reordered by the CPU/optimizer. We need to add memory barriers to mitigate this. -
Large memcpy window: We are copying a sufficiently large memory region via
memcpy. If two or more GC cycles occur during this copy, we can end up with inconsistent data spanning multiple GC states.
@maurycy has an idea to use a sequence counter for consistency checks — I think he should definitely give it a try.
cc @pablogsal
CPython versions tested on:
CPython main branch
Operating systems tested on:
No response
Linked PRs
- gh-155828
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing get_gc_stats and the locations that store ts_start and ts_stop, then review the memcpy path for how an external process observes GC data. Read linked PR gh-155828 and any related tests first; done means external sampling no longer returns snapshots torn across GC cycles on weakly ordered platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100