Prevent pyo3 rebuilds with `cargo`, `maturin`, `rust-analyzer`, ... through generated build config file
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 16.2k
- Forks
- 1k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 66
Description
When, in one crate, Python extension modules are build (with maturin), and Rust executables (with cargo), whenever launching the other build, pyo3 is rebuilt too.
This is triggered for me because maturin sets:
PYO3_ENVIRONMENT_SIGNATURE=cpython-3.13-64bit
PYO3_BUILD_EXTENSION_MODULE=1
But cargo build has both env vars unset. Same goes for IDE-run rust-analyzer, ... so it rebuilds 😢.
So this is a different take on #1708.
❓ Workaround: use something like direnv and put the env vars above in .envrc so they are set for cargo and rust-analyzer and won't change between runs (determine them with CARGO_LOG=cargo::core::compiler::fingerprint=info cargo build after running maturin).
⭐ Real solution proposal: Move away from the env-vars approach.
- I think the current approach needs to be re-thought:
- Pyo3 reconfiguration is only triggered by environment changes (
rerun-if-env-changed), but by default the env is (and should be) empty => Using the environment as signaling method will never get rid of unintended rebuilds if not all tools set the exact same envirnment as signals as well.
- Pyo3 reconfiguration is only triggered by environment changes (
- How about using
rerun-if-changedwith files instead:- instead of syncing an environment variable name, use a common generated configuration file (e.g.
$OUT_DIR/pyo3_buildconfig.toml?) - each tool aware of it, either adjusts it or leaves it unchanged if it discovered/decided on the same variable value
- any tool invocation like
rust-analyzerdoesn't touch it anyway, so we don't have a rebuild
- instead of syncing an environment variable name, use a common generated configuration file (e.g.
🟢 So in other words: write config file(s) instead of env vars, because they remain the same/unchanged if tools are not aware of them. Tools that aware of them are expected to adjust them, and then trigger rebuilds.
What do you think?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining how maturin, cargo, and rust-analyzer use PYO3_ENVIRONMENT_SIGNATURE and PYO3_BUILD_EXTENSION_MODULE during builds. Compare their rebuild behavior, then work out the generated configuration-file approach and its change conditions; done means unchanged invocations no longer rebuild pyo3 while intended configuration changes still do.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- build-system, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100