lablup / lablup/backend.ai

Encrypted secret-key storage with self-describing encryption method

Open
#13,006 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

Secret keys and credentials (keypair secret_key/ssh_private_key, object storage & reservoir secret_key/access_key, container registry password, huggingface token, etc.) are currently stored in plain text (plain sa.String/sa.Text). There is no encryption-at-rest helper in common/ or manager/.

Goal: introduce a reversible EncryptedColumn (TypeDecorator in models/base.py) that stores the encryption method alongside the value, analogous to how PasswordColumn stores algorithm$rounds$salt$hash. This enables an incremental rollout of at-rest encryption:

1. plain text (legacy back-compat; rows without a prefix are interpreted as plain)
2. config-injected key-based encryption (master key from local_config/etcd)
3. KMS-based encryption (later; provider abstraction)

Design direction (to be detailed in the BEP):
- Single encryption scheme: symmetric AEAD (e.g. AES-256-GCM). Encryption algorithm and on-disk format are uniform across config and KMS.
- Envelope encryption: payload is always encrypted locally with a DEK; only DEK wrapping/unwrapping differs. A pluggable KeyProvider (noop/config/KMS) resolves the key – this is the only branch point between modes.
- Self-describing stored format carries scheme/version/algorithm + a key_ref (key id/alias, NOT the key material) + nonce + wrapped DEK + ciphertext. No separate plaintext hash: the AEAD auth tag already detects wrong-key/tampering on decrypt. Key material is never stored in the column.

This is a BEP-first epic: the BEP (Task #2) must be written and accepted before implementation Stories are decomposed via /bep-to-issues.

JIRA Issue: BA-6958

Contributor guide

Open the contributing guide

Research direction

Start with the encryption-related model conventions in models/base.py and review the requested BEP-first process. Define the accepted BEP for the self-describing encrypted format, key-provider modes, legacy plaintext handling, and rollout boundaries. Done means the BEP is written and accepted before implementation stories are decomposed via /bep-to-issues.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.