python / python/cpython

Torn reads of GC statistics from external process via get_gc_stats

Open
#155,811 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

3.15 3.16 extension-modules interpreter-core type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

If an external tool sampling GC statistics does not pause the target process (and we designed get_gc_stats not to pause), it can read torn data under certain circumstances.

There are two distinct problems:

  1. Weakly ordered platforms: The stores of ts_start and ts_stop may be reordered by the CPU/optimizer. We need to add memory barriers to mitigate this.

  2. Large memcpy window: We are copying a sufficiently large memory region via memcpy. If two or more GC cycles occur during this copy, we can end up with inconsistent data spanning multiple GC states.

@maurycy has an idea to use a sequence counter for consistency checks — I think he should definitely give it a try.

cc @pablogsal

CPython versions tested on:

CPython main branch

Operating systems tested on:

No response

Linked PRs
  • gh-155828

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing get_gc_stats and the locations that store ts_start and ts_stop, then review the memcpy path for how an external process observes GC data. Read linked PR gh-155828 and any related tests first; done means external sampling no longer returns snapshots torn across GC cycles on weakly ordered platforms.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.