Track a future SandboxExecutor for isolated Task SDK workloads
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
## Scope
This issue tracks a **future `SandboxExecutor`** that would run a complete Airflow Task SDK `ExecuteTask` workload in an isolated, ephemeral runtime.
It does **not** track the narrower agent code-execution tool proposed by #68847, and #68847 does not close this issue.
## Sandboxed-execution surfaces
These are separate execution boundaries, not one API and not a monotonic security ranking:
| Surface | What moves into the sandbox | Airflow role | Status |
|---|---|---|---|
| **Restricted code orchestration** | Monty-compatible glue code only | `AgentOperator(code_mode=True)` orchestrates registered worker-side tools | Implemented by #68407 |
| **Sandboxed code execution as a tool** | Model-generated full Python | The credentialed agent remains outside and calls a sandbox as a tool | Proposed by #68847 |
| **Sandboxed command/job execution** | One author-defined command or job | A future resumable/deferrable `SandboxOperator` could manage files, artifacts, cancellation, and reconnection | Tracked by #69862 |
| **Sandboxed TaskInstance execution** | The complete Task SDK `ExecuteTask` workload | A `SandboxExecutor` becomes responsible for remote task lifecycle | Tracked by this issue |
Moving more of the task into a sandbox is not automatically safer for every use case. A whole-task executor normally provides the Task SDK workload with scoped Airflow API access, while sandbox-as-tool can deliberately keep the agent's credentials outside the generated-code environment.
## Future SandboxExecutor responsibilities
A sandbox runtime's `create/run/destroy` operations are not, by themselves, an Airflow executor contract. A production executor must additionally handle:
- deterministic workload identity and idempotent submission;
- capacity, queueing, heartbeat, and terminal-state reconciliation;
- Task SDK token and DAG-bundle delivery;
- durable task logs and result state;
- cancellation and forced termination;
- scheduler-restart adoption and external resource lookup;
- sandbox disappearance, ambiguous provider responses, and retry safety;
- leases, TTLs, and leaked-resource reaping;
- files, artifacts, secrets, network policy, and provider trust boundaries.
The implementation should use Airflow's existing public `BaseExecutor` and `ExecuteTask` interfaces rather than treating the agent-focused backend in #68847 as the executor SPI.
## Motivation
Potential use cases include:
- isolated per-task dependencies and runtime images;
- bursty or heavyweight jobs without a shared worker image;
- stronger infrastructure boundaries for sensitive workloads;
- ephemeral remote compute without requiring every installation to operate Celery or Kubernetes workers.
A whole-task executor is not the same security solution as isolating only model-generated code. Both may eventually be useful and composable.
## Proposed progression
1. Harden and evaluate the narrowly scoped `SandboxToolset` in #68847.
2. Establish backend conformance tests and clear vendor ownership.
3. Prove files, artifacts, cancellation, reconnection, and lifecycle semantics through an operator-level integration.
4. Develop a third-party `SandboxExecutor` against `BaseExecutor` and `ExecuteTask`.
5. Extract broader shared runtime types only after the tool, operator, and executor consumers demonstrate genuinely common semantics.
## Related work
- #68407 — merged Monty-based restricted code orchestration.
- #68847 — proposed sandbox-as-tool integration in `common.ai`.
- https://github.com/zozo123/airflow-provider-sandbox — experimental whole-task executor prototype; not the implementation of #68847.
Contributor guide
Assessment
This issue has not been assessed yet.