python / python/cpython

compileall shouldn't recompile up-to-date files with hash-based invalidation mode

Open
#121,577 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

stdlib type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

When you compile sources into .pyc files with python -m compileall while using the defaul invalidation mode (timestamp), the script verifies whether the compilation is necessary (the pyc file is up to date) and can skip it:

$> python -m compileall --invalidation-mode=timestamp script.py
Compiling 'script.py'...
$> python -m compileall --invalidation-mode=timestamp script.py
$>   # the compilation is skipped

When either of the hash invalidation modes is used instead, this doesn't work - the hash based check is not there. https://github.com/python/cpython/blob/main/Lib/compileall.py#L229

$> python -m compileall --invalidation-mode=checked-hash script.py
Compiling 'script.py'...
$> python -m compileall --invalidation-mode=checked-hash script.py
Compiling 'script.py'...
$>

I think it would make sense to add the same functionality for the other two invalidation modes as well?

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs
  • gh-121960

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Lib/compileall.py at the invalidation-mode handling around the linked line, and reproduce the behavior with the two python -m compileall commands shown for checked-hash and unchecked-hash modes. Done means repeated compilation of an up-to-date source is skipped for both hash invalidation modes, while preserving the existing timestamp behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.