lablup / lablup/backend.ai

Announce storage backends and volumes in the storage proxy heartbeat

Open
#14,172 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

The service catalog already records a service identity, its endpoints and its liveness, and the storage proxy already publishes into it. What the record does not carry is the storage-specific declaration: which backend appliances this proxy is configured against, and which volumes it mounts. Without it the manager cannot build the storage records described in BEP-1078.

Extend the storage proxy heartbeat payload to declare:

- Backends: name, type, and the capability flags the backend supports (vfolder, metric, quota, fast filesystem size, fast scan, fast size).
- Volumes: name, the name of the backend that hosts it, and this proxy's mount path for it.

The volume configuration gains an optional backend name. It stays optional so that existing configuration files keep working: when it is absent the backend type is used as the name, and such an installation registers a backend named after its type.

Connection details and credentials must never be included. The heartbeat travels over the shared event bus, which every component consuming events can read; those values are set through the administrator API instead.

Out of scope: consuming the payload on the manager side, and moving the manager's storage proxy client onto catalog-resolved endpoints. The latter is covered by the unified service discovery epic.

Design: BEP-1078 (Storage Proxy Enhancement). Upstream: BEP-1046 (Unified Service Discovery with DB-backed Service Catalog).

JIRA Issue: BA-7630

Contributor guide

Open the contributing guide

Research direction

Start by tracing the storage proxy heartbeat implementation and the volume configuration handling, then read BEP-1078 for the backend and volume declaration requirements. Verify the heartbeat contains only backend capabilities and volume mount metadata, preserves configurations without a backend name, and excludes credentials and connection details; manager-side consumption is out of scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.