lablup / lablup/backend.ai

Add named secrets with immutable versions

Open
#13,984 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

Implements phase 1 of BEP-1074 (proposals/BEP-1074-container-secret-references.md).

Adds two tables: secrets (a shell holding a name and an owning scope) and secret_versions (where the value lives, immutable). A value is changed by adding a version, never by updating one. Scope is user, project or domain.

Also corrects an existing declaration: BA-7044 introduced SECRET_ENTITY_TYPE purely to satisfy the abstract entity_type() of a global action. Re-encryption crosses several kinds of encrypted column, so it names no entity kind, and actions/AGENTS.md states GLOBAL_ENTITY_TYPE is what such an operation records. The two re-encryption actions return GLOBAL_ENTITY_TYPE instead, and common/data/entity/secret.py is rewritten as the real entity declaration (SECRET_ENTITY_TYPE, SECRET_SCOPE_TYPE, SecretID). BaseGlobalAction.entity_type() stays abstract; the AGENTS.md sentence about never referencing GLOBAL_ENTITY_TYPE is corrected to match actual behavior.

Commit split for review:
1. Entity type correction plus both table schemas and their v2 specs
2. Repository and service operations, plus the re-encryption target catalog entry
3. REST v2 and GraphQL surfaces
4. SDK v2 and CLI v2

Key design points:
- Version numbers are computed inside the INSERT (a scalar subquery over the secret's highest number) rather than read first, so no row lock is needed; the unique constraint rejects a concurrent loser and the caller retries. This mirrors deployment_revisions.revision_number.
- No update spec on secret_versions. The update action carries no value field, which is what enforces version immutability at the API.
- SecretData and SecretVersionData carry no value field, so no read path can return one.
- secrets carries created and updated timestamps; secret_versions carries created only.
- The CLI takes values from stdin or a prompt, never as a command-line argument.
- No automatic version cleanup and no retention policy category.

Open question to settle before starting: whether creating a project or domain secret takes an admin of that scope or any member.

See the draft for the full file and feature tables: drafts/BEP-1074-container-secrets/issues.md

Success Criteria
- [ ] Both tables are created by a migration and the downgrade works
- [ ] A duplicate name within one scope is rejected by the unique constraint, and a scope_id that does not match its scope_type is rejected by the check constraint
- [ ] Concurrent version additions to one secret produce consecutive, non-colliding numbers
- [ ] The audit and reporter records of the re-encryption actions carry global rather than secret
- [ ] Creating a secret also creates version 1, and no return value carries the original value
- [ ] The update action renames a secret and offers no argument that changes a value
- [ ] Adding a version increments the number and leaves the previous version intact
- [ ] Deleting a secret that still has versions is refused
- [ ] With a write provider configured the stored value is ciphertext; with plain it is plaintext
- [ ] A re-encryption run covers secret_versions.value
- [ ] No REST v2, GraphQL or CLI read output carries a value
- [ ] Secrets of another scope do not appear in a listing
- [ ] The CLI does not accept a value as a command-line argument
- [ ] Schema dumps are regenerated and the inspector passes
- [ ] pants test passes for affected packages

JIRA Issue: BA-7495

Contributor guide

Open the contributing guide

Research direction

Start with proposals/BEP-1074-container-secret-references.md and drafts/BEP-1074-container-secrets/issues.md, then inspect common/data/entity/secret.py and actions/AGENTS.md. Trace the four review commits across the schemas, repository, service, REST, GraphQL, SDK, and CLI surfaces. Done means the listed success criteria pass, including migrations, immutable versions, scope isolation, redacted outputs, re-encryption coverage, regenerated schema dumps, and affected-package tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
graphql, python
Domain
api, backend-api-design, cli, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.