Local External Python Modules

Outcome

Allow a DimOS module author to run a separately packaged local Python implementation in its own managed dependency environment while it remains an ordinary Blueprint participant. Authors declare one implementation import reference; Blueprint composition, typed streams, RPC, skills, module references, configuration, and restart behavior remain unchanged.

System Shape

Primary shape: pipeline. The implementation crosses the declaration, worker, runtime bootstrap, and normal coordinator lifecycle.

Blueprint declaration
  configuration + In/Out streams + RPC/skills + module references
  implementation = "package.module:RuntimeClass"
                     │
                     ▼
ModuleCoordinator deploys through WorkerManagerPython
                     │
                     ▼
Private external-Python worker
  1. resolves <declaration-dir>/python/
  2. validates pyproject.toml and detects optional pixi.toml
  3. prepares the managed runtime environment
  4. launches a bootstrap in that environment
                     │
                     ▼
Runtime bootstrap
  imports implementation → validates declaration contract
  → serves RPC under declaration identity
                     │
                     ▼
Normal coordinator lifecycle
  connect streams → inject module references → build → start

The worker returns the existing coordinator-facing RPC proxy shape. Nothing above WorkerManagerPython needs to know that the implementation uses a separate Python environment.

Final API Shape

my_feature/
├── declaration.py
└── python/
    ├── pyproject.toml
    ├── pixi.toml                   # optional
    └── my_feature_runtime/
        └── runtime.py

# declaration.py — lives in the host DimOS environment
class MyFeature(ExternalPythonModule):
    implementation = "my_feature_runtime.runtime:MyFeatureRuntime"

    input: In[Image]
    output: Out[Result]
    _dependency: DependencySpec

    @skill
    def enable(self, enabled: bool) -> str:
        """Enable or disable this feature."""
        ...


# python/my_feature_runtime/runtime.py — lives in the runtime project
class MyFeatureRuntime(MyFeature):
    def start(self) -> None: ...
    def _process(self, image: Image) -> None: ...


# blueprint.py — no external-runtime API
stack = autoconnect(MyFeature.blueprint(), consumer.blueprint())

ExternalPythonModule is the declaration base. Its only external-runtime field is implementation; all normal module configuration, streams, RPC methods, skills, and references stay on the declaration. The runtime extends that declaration, so the bootstrap can validate one contract and serve it under the MyFeature identity.

At deployment, the worker resolves declaration.py's sibling python/ project. pyproject.toml is mandatory and declares all runtime Python dependencies. With no pixi.toml, DimOS prepares and runs the project through uv. With one, it runs uv through Pixi; Pixi provides the outer tool environment and does not replace the uv project. PYTHONPATH is not used for dependencies. Preparation, import, or contract failures abort deployment before module startup.

Synced from DIM-1222 by che

Start by tracing how ModuleCoordinator deploys through WorkerManagerPython and how the runtime bootstrap validates and serves a module. Review the declaration.py, blueprint.py, pyproject.toml, and optional pixi.toml roles described in the issue. Done means an external Python implementation can deploy in its managed environment while preserving normal streams, RPC, skills, references, configuration, and restart behavior.