Migrate storage configuration from etcd to the database
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
Once the normalized storage tables exist and services declare their backends and volumes, move the existing data off etcd and retire what it leaves behind.
Scope:
- Move the existing configuration into the database through a manager CLI command rather than an Alembic migration, so that it can be rehearsed with a dry run and repeated. Volume and backend records are pulled from the running storage proxies, because etcd holds no volume data. The resource group to volume table is seeded with every group and every volume, since resource groups place no restriction on volumes today and without the seed no session could mount anything afterwards.
- Retire the etcd volumes configuration schema, its loader, and the helpers that parse the proxy and volume names out of the virtual folder host string.
- Enforce the storage volume reference on virtual folders. The column is added nullable with the schema; once the migration has run it can be constrained, exempting unmanaged folders, which point at a raw path rather than a volume, and folders in a terminal deletion state.
Design: BEP-1078 (Storage Proxy Enhancement).
JIRA Issue: BA-7655
Contributor guide
Research direction
Start with BEP-1078, the manager CLI, and the existing etcd volumes configuration loader. Trace the normalized storage tables, running storage proxy data, virtual-folder schema, and helpers that parse proxy and volume names. Done means a repeatable dry-run migration seeds the database correctly, retires the old etcd path, and enforces the volume reference for eligible folders.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cli, database
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100