RustPython / RustPython/RustPython

[RFC] Bundle frozen modules

Open
#4,204 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

RFC
Dominant language
Rust
Stars
22.4k
Forks
1.5k
Avg merge
16h 33m
Merged PRs (30d)
165

Description

Summary

Right now my understanding is that freezing (such as with py_freeze) just takes a file or directory as input and naively freezes all .py files found, basically speaking. I suggest that freezing follow static imports to automatically find and freeze modules located on the user's machine.

Detailed Explanation

While building Kybra we have come across the very basic need to include all of our user's Python code into a final Wasm binary to deploy to our decentralized cloud environment (the Internet Computer). We currently freeze all of the user's modules at compile time, and we use py_freeze to achieve this. Unfortunately py_freeze just takes a file or directory as input. But we want to be able to freeze a user's entire Python application, which would include all imported modules, and the transitive imports of those imported modules.

We have experimented with our own module bundler based on modulegraph to achieve something like this. It is working rather well during our initial prototyping. But it would be even better to not have to build and maintain this tool outside of RustPython. It would also be nice for RustPython to include tree shaking and/or dead code elimination as discussed in this RFC.

Drawbacks, Rationale, and Alternatives

This type of module bundling would only work for static imports, so anything dynamically imported would have to be included in other ways. The alternative I've been using is to create my own bundler. To me it seems obvious that py_freeze would be able to freeze a module and all of its dependencies, without requiring them to all be in the same directory.

Unresolved Questions

If there's a better way to achieve this then that would be great, but I think the idea is pretty simple. When freezing follow the imports and freeze all transitive modules.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the py_freeze entry point and trace how it currently discovers files, then compare the proposed static-import traversal with the modulegraph approach mentioned in the issue. Done means freezing follows imports transitively across modules outside the initial directory; dynamically imported modules remain outside this scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust, wasm
Domain
build-system, compilers
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.