indygreg / indygreg/PyOxidizer
Feature Request: Dynamically linked bundle of classified resources
- Dominant language
- Rust
- Stars
- 6.2k
- Forks
- 256
- PR merge metrics
- No merged PRs in 30d
Description
In a typical Python scientific computing project, the on-disk size is probably dominated by large dependencies like `scipy` and `numpy` (88 and 46 MB in my build, respectively), not to mention the standard library. These dependencies likely change at a far slower rate than the local Python and Rust code that makes up the rest of the project. Having to redistribute a 100 MB+ bundle every update is less than ideal.
Granted, those large dependencies are also `__file__` users, and must be stored in a "lib" folder anyway. For now, it's possible to leave those alone and distribute the smaller binary (39 MB in my build) instead. However, that binary still contains the stdlib and other packages that can be classified, because I'd like to make use of the fast, zero-copy imports where I can. In future, once the package resources problem is (more) resolved, (more of) those dependencies will make it into the binary to enjoy the same benefits. This is key, I think: the benefit from packing dependencies into the binary is not _just_ "single file" distribution.
It would be really neat if we could package rarely-changing dependencies into a "dependencies.dll" file, or similar, to preserve fast imports while reducing update size. Dynamic linking being what it is, I imagine that this would be very nontrivial to implement cross-platform. Or maybe not; this is outside my area of expertise. Just wanted to raise it as a suggestion!
Contributor guide
No contributing guide indexed for this repository
Research direction
No files, tests, or entry points are named in the issue. Start by determining how PyOxidizer currently classifies and packages Python and Rust resources; done would mean a cross-platform design or implementation for dynamically linked dependency bundles that preserves fast imports while reducing update sizes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100