RustPython / RustPython/RustPython
[RFC] Bundle frozen modules
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 22.4k
- Forks
- 1.5k
- Avg merge
- 16h 33m
- Merged PRs (30d)
- 165
Description
Summary
Right now my understanding is that freezing (such as with py_freeze) just takes a file or directory as input and naively freezes all .py files found, basically speaking. I suggest that freezing follow static imports to automatically find and freeze modules located on the user's machine.
Detailed Explanation
While building Kybra we have come across the very basic need to include all of our user's Python code into a final Wasm binary to deploy to our decentralized cloud environment (the Internet Computer). We currently freeze all of the user's modules at compile time, and we use py_freeze to achieve this. Unfortunately py_freeze just takes a file or directory as input. But we want to be able to freeze a user's entire Python application, which would include all imported modules, and the transitive imports of those imported modules.
We have experimented with our own module bundler based on modulegraph to achieve something like this. It is working rather well during our initial prototyping. But it would be even better to not have to build and maintain this tool outside of RustPython. It would also be nice for RustPython to include tree shaking and/or dead code elimination as discussed in this RFC.
Drawbacks, Rationale, and Alternatives
This type of module bundling would only work for static imports, so anything dynamically imported would have to be included in other ways. The alternative I've been using is to create my own bundler. To me it seems obvious that py_freeze would be able to freeze a module and all of its dependencies, without requiring them to all be in the same directory.
Unresolved Questions
If there's a better way to achieve this then that would be great, but I think the idea is pretty simple. When freezing follow the imports and freeze all transitive modules.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the py_freeze entry point and trace how it currently discovers files, then compare the proposed static-import traversal with the modulegraph approach mentioned in the issue. Done means freezing follows imports transitively across modules outside the initial directory; dynamically imported modules remain outside this scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust, wasm
- Domain
- build-system, compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100