e2b-dev / e2b-dev/runtime

feat: template build retention policy — auto-clean stale CI builds

Open
#3,197 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1.6k
Forks
438
PR merge metrics
No merged PRs in 30d

Description

Problem

Every call to e2b template build creates a new build record and leaves the previous one around indefinitely. Teams running CI/CD pipelines (e.g. rebuilding a template on every commit) accumulate hundreds of stale builds with no automatic cleanup path.

The current state:

  • Templates / builds are permanent until a user explicitly deletes them
  • There is no per-team template count limit in the tiers table (only concurrent_template_builds for in-progress builds, not stored ones)
  • last_spawned_at and spawn_count are tracked in the envs table but nothing reads them for lifecycle management

This creates three compounding issues:

  1. Unbounded storage growth — object-storage costs grow silently for active CI teams
  2. Operational noise — dashboard and API responses list dozens of irrelevant historical builds
  3. No quota enforcement — unlike sandbox runtime (max_length_hours) and concurrency (concurrent_instances), template storage has no tier-level control

Proposed solution

Add a configurable build retention policy that can be set at the team or template level:

1. Keep-last-N policy (highest priority)

Retain the N most recent successful builds for a template; soft-delete older ones automatically.

PATCH /templates/{templateID}/retention
{ "keep_last_builds": 5 }

This directly addresses the CI use case: a pipeline that builds 100 times keeps only the last 5, without the user having to remember to clean up.

2. Inactivity-based policy (optional, opt-in)

Soft-delete builds not spawned in the last N days. Safe because:

  • Uses the existing last_spawned_at column — data is already tracked
  • Operates on deleted_at soft-delete — fully reversible within a grace period
  • Must be explicitly opted-in (never a default) to avoid breaking active-but-infrequent workloads
3. Per-tier template count cap (enforcement)

Add max_stored_templates to the tiers table (following the same pattern as concurrent_template_builds added in #3097). This gives platform operators a hard ceiling for storage growth per team.

What already exists (no schema changes needed for the core feature)

Existing piece Role in this feature
envs.last_spawned_at inactivity policy threshold
envs.spawn_count activity signal for ranking
envs.deleted_at soft-delete (reversible cleanup)
active_envs view already filters deleted_at IS NULL — all read paths are safe
tiers table add max_stored_templates column following existing pattern

What needs to be built

  • Background job (cron or event-driven) that evaluates retention rules and stamps deleted_at
  • max_stored_templates column in tiers + enforcement in the template registration path
  • API endpoint to set per-template retention config
  • Dry-run mode: list what would be deleted before committing
  • Dashboard UI to show and configure retention settings

Non-goals

  • Hard TTL on templates (absolute expiry regardless of usage) — too risky for production workloads that use templates infrequently
  • Deleting the currently-active build of a template — retention should only clean up builds that are not the latest pinned version

Prior art

  • Docker Hub: keeps last N tags, warns on storage approaching quota
  • GitHub Actions: configurable artifact retention (default 90 days, overridable per workflow)
  • AWS ECR: lifecycle policies (keep last N images, expire by age)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the e2b template build entry point and the template registration path, then trace the envs, tiers, and active_envs data paths. Compare the requested retention endpoint, background cleanup, quota enforcement, dry-run behavior, and dashboard configuration against existing patterns. Done means all listed pieces work without deleting the latest pinned build.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
api, backend, database, devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.