e2b-dev / e2b-dev/runtime

Title: feat: add bin-packing scheduling strategy for sandbox placement

Open
#3,104 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1.6k
Forks
438
PR merge metrics
No merged PRs in 30d

Description

Summary

Add an optional bin-packing scheduling strategy alongside the existing spread (Best-of-K) strategy. This allows operators to choose between two placement modes via a feature flag:

  • Spread (default): Distribute sandboxes evenly across all nodes — best for high availability and fault isolation
  • Bin-pack: Fill one node before moving to the next — best for cost optimization and resource consolidation
Motivation

The current Best-of-K placement algorithm always prefers nodes with lower utilization (spread strategy). This is ideal for large-scale deployments where fault isolation matters, but suboptimal for:

  1. Cost-sensitive self-hosted deployments: When running on auto-scaling infrastructure, bin-packing keeps fewer nodes active, allowing idle nodes to be scaled down and reducing cloud costs
  2. Small clusters (2-5 nodes): Spreading 10 sandboxes across 5 nodes means all 5 nodes must stay running. Bin-packing would concentrate them on 1-2 nodes, freeing the rest
  3. Dev/staging environments: Where availability is less critical than cost
Proposed Design
  • Add a feature flag sandbox-placement-bin-pack (default: false)
  • When enabled, reverse the scoring logic in BestOfK.Score(): prefer nodes with higher utilization (more allocated resources)
  • In bin-pack mode, evaluate all eligible nodes instead of sampling K random ones, ensuring the most-loaded node is always selected
  • No API changes required — purely a server-side scheduling optimization
Behavior Comparison

Scenario (3 nodes, 6 sbxs) Spread Bin-pack
Node A 2 sandboxes 4 sandboxes
Node B 2 sandboxes 2 sandboxes
Node C 2 sandboxes 0 (can scale down)

/cc @jakubno

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the existing BestOfK.Score() implementation and the sandbox placement feature-flag configuration. Read how eligible nodes are sampled and scored, then run the scheduler tests that cover placement behavior. Done means the default spread strategy is unchanged, bin-pack evaluates all eligible nodes and prefers higher utilization, and the feature flag remains disabled by default.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.