Reference secrets from sessions and inject them into containers
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
Implements phase 2 of BEP-1074 (proposals/BEP-1074-container-secret-references.md). Depends on the secrets entity.
Adds secret_bindings, a field row of sessions and deployment revisions holding (owner, env_key, version_id). A binding pins a specific version, so rotating a value never changes what an existing session receives. This issue writes only session-owned bindings; revisions come in phase 3.
A create request may name a secret, and the service pins the latest version at that moment; naming a version directly is also accepted, which reproducible deployments need.
Commit split for review:
1. secret_bindings schema, session create and read APIs, cleanup paths
2. Delivery path: decryption, the RPC split, agent merge and masking
Key design points:
- version_id is a real foreign key with ON DELETE RESTRICT, so a referenced version cannot be deleted. INDEX(version_id) serves the reverse lookup that answers a leaked value. The owner side spans two types, so it is an enumerated column cleaned up by code.
- Name collisions are refused at creation: a name already present in environ, and any name starting with BACKENDAI_. Do not define a merge precedence, which would silently overwrite.
- Two session deletion paths need cleanup. The v2 delete goes through _teardown_entity; the retention policy bulk delete does not and takes its own entry.
- Secrets never enter the session draft's environ, so sessions.environ and kernels.environ stay free of them without anything blocking it.
- The RPC payload keeps secrets in a field separate from environ for one reason: log masking. The values merge into environ at the agent anyway. The config dump happens before the merge, and that ordering is what makes masking work, so pin it with a comment.
- After merging, remove the secret field from the config. The whole config is serialized for restarts immediately afterward into a directory the container reads, so leaving it in stores the values twice.
- Docker Env stays as it is. Do not add a file-based delivery path or a tmpfs mount; BEP-1074 leaves both open.
See the draft for the full file and feature tables: drafts/BEP-1074-container-secrets/issues.md
Success Criteria
- [ ] Naming a secret at session creation records the version that was latest at that moment
- [ ] Adding a new version afterwards leaves that session's binding on the old version
- [ ] A session can be created against a directly named version
- [ ] Naming a secret the requester cannot read is refused
- [ ] A name colliding with environ and a BACKENDAI_ prefix are both refused
- [ ] A session read answers with environment variable names and origins, never values
- [ ] Deleting a referenced version is refused
- [ ] Deleting a session through v2 or through the retention policy leaves no binding behind
- [ ] No secret appears in sessions.environ or kernels.environ
- [ ] The environment variable holds the original value inside the container of a session carrying a secret
- [ ] With log-kernel-config on, the log carries key names and no values
- [ ] The restart config stored for the kernel does not carry the same values twice
- [ ] Restarting a kernel injects the same value
- [ ] Sessions without secrets behave as before
- [ ] pants test passes for affected packages
JIRA Issue: BA-7496
Contributor guide
Research direction
Start with proposals/BEP-1074-container-secret-references.md and drafts/BEP-1074-container-secrets/issues.md, then trace session create/read APIs, _teardown_entity, the retention-policy bulk-delete entry, and the agent delivery path. Implement and verify the listed binding, cleanup, masking, restart, and container-injection criteria, then run pants test for affected packages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- api, backend, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100