lablup / lablup/backend.ai

Reference secrets from sessions and inject them into containers

Open
#13,985 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Implements phase 2 of BEP-1074 (proposals/BEP-1074-container-secret-references.md). Depends on the secrets entity.

Adds secret_bindings, a field row of sessions and deployment revisions holding (owner, env_key, version_id). A binding pins a specific version, so rotating a value never changes what an existing session receives. This issue writes only session-owned bindings; revisions come in phase 3.

A create request may name a secret, and the service pins the latest version at that moment; naming a version directly is also accepted, which reproducible deployments need.

Commit split for review:
1. secret_bindings schema, session create and read APIs, cleanup paths
2. Delivery path: decryption, the RPC split, agent merge and masking

Key design points:
- version_id is a real foreign key with ON DELETE RESTRICT, so a referenced version cannot be deleted. INDEX(version_id) serves the reverse lookup that answers a leaked value. The owner side spans two types, so it is an enumerated column cleaned up by code.
- Name collisions are refused at creation: a name already present in environ, and any name starting with BACKENDAI_. Do not define a merge precedence, which would silently overwrite.
- Two session deletion paths need cleanup. The v2 delete goes through _teardown_entity; the retention policy bulk delete does not and takes its own entry.
- Secrets never enter the session draft's environ, so sessions.environ and kernels.environ stay free of them without anything blocking it.
- The RPC payload keeps secrets in a field separate from environ for one reason: log masking. The values merge into environ at the agent anyway. The config dump happens before the merge, and that ordering is what makes masking work, so pin it with a comment.
- After merging, remove the secret field from the config. The whole config is serialized for restarts immediately afterward into a directory the container reads, so leaving it in stores the values twice.
- Docker Env stays as it is. Do not add a file-based delivery path or a tmpfs mount; BEP-1074 leaves both open.

See the draft for the full file and feature tables: drafts/BEP-1074-container-secrets/issues.md

Success Criteria
- [ ] Naming a secret at session creation records the version that was latest at that moment
- [ ] Adding a new version afterwards leaves that session's binding on the old version
- [ ] A session can be created against a directly named version
- [ ] Naming a secret the requester cannot read is refused
- [ ] A name colliding with environ and a BACKENDAI_ prefix are both refused
- [ ] A session read answers with environment variable names and origins, never values
- [ ] Deleting a referenced version is refused
- [ ] Deleting a session through v2 or through the retention policy leaves no binding behind
- [ ] No secret appears in sessions.environ or kernels.environ
- [ ] The environment variable holds the original value inside the container of a session carrying a secret
- [ ] With log-kernel-config on, the log carries key names and no values
- [ ] The restart config stored for the kernel does not carry the same values twice
- [ ] Restarting a kernel injects the same value
- [ ] Sessions without secrets behave as before
- [ ] pants test passes for affected packages

JIRA Issue: BA-7496

Contributor guide

Open the contributing guide

Research direction

Start with proposals/BEP-1074-container-secret-references.md and drafts/BEP-1074-container-secrets/issues.md, then trace session create/read APIs, _teardown_entity, the retention-policy bulk-delete entry, and the agent delivery path. Implement and verify the listed binding, cleanup, masking, restart, and container-injection criteria, then run pants test for affected packages.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
api, backend, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.