indygreg / indygreg/PyOxidizer

Ditch Starlark for Python?

Open
#444 11 comments 6 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
6.2k
Forks
256
PR merge metrics
No merged PRs in 30d

Description

PyOxidizer's configuration files were originally TOML. Then we switched to Starlark so we could leverage a _real_ programming language to express complex configuration. Starlark was chosen over alternatives like Lua because it is purposefully designed to be a configuration language and it has great sandboxing properties.

Maintaining the Starlark code in PyOxidizer has historically been a pain. The crate was effectively dormant for a while. Then Facebook took it over. While they have done great work with the crate, it doesn't support building without Rust Nightly features (although they've offered to fix that once my port past version 0.3 of the crate is in a good position).

As I look at the sheer amount of effort that it will take to port off version 0.3 of the Starlark crate, I question whether the effort would be better spent porting to actual Python instead. Either embed a CPython interpreter in `pyoxidizer` or a Rust implementation of Python like https://github.com/RustPython/RustPython. The latter would be preferred, as CPython's embedding/sandboxing story isn't great and we'd effectively open up PyOxidizer configuration files to a fully-featured scripting environment. I kind of like keeping behavior more tightly constrained, even if that does mean we end up reinventing a few wheels for things like file I/O.

A benefit to using actual Python is that the configuration files will be more friendly to Python developers. We can do things like a ship a `.pyi` file defining the typing interface so autocomplete works in IDEs and configuration files can be type validated. This also opens us up to a Python API for PyOxidizer, enabling people to create their own tools leveraging PyOxidizer's internals for building binaries.

If anyone has any thoughts on embedding Python interpreters in Rust, feel free to leave comments here. I'm most curious about sandboxing. CPython can't sandbox Python code that well. Do Python implementations in Rust (like RustPython) enable you to do things like limit which `builtins` are exposed and prevent explicit file I/O from the interpreter?

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files, tests, or entry points. Start with the 11-comment discussion and investigate CPython versus RustPython embedding and sandboxing; a concrete migration scope and acceptance criteria are still needed before work can be considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
build-system, devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.