lablup / lablup/backend.ai

Add a usage-free per-id vfolder metadata endpoint (avoid the Storage-Proxy tree-walk on `GET /folders/{id}`)

Open
#12,834 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Problem

There is no cheap way to fetch a **single** vfolder's control-plane metadata by id. `GET /folders/{id}` (get_info) unconditionally triggers a Storage-Proxy `folder/usage` call that performs a **full recursive tree walk** on vfs-family backends, making every per-id lookup O(folder size) and adding a transitive runtime dependency on Storage-Proxy availability. Satellite services are therefore forced to resolve buckets via the full `GET /folders` **list** and filter client-side, even when they want just one folder's record.

This spike confirmed the two list-shaped resolvers (`GET /folders`, `POST /v2/vfolders/my/search`) are DB-only, while `GET /folders/{id}` is the one that scans — see [docs/spikes/03-bucket-resolver.md](../03-bucket-resolver.md).

## Request

A **usage-free** per-id metadata endpoint (or a `?include_usage=false` / metadata-only mode on `GET /folders/{id}`) that returns the DB-backed control-plane fields — `id, host, quota_scope_id, permission, status, is_owner, ownership_type, created_at, name` — **without** the Storage-Proxy usage scan.

## Rationale

- Lets satellite gateways do O(1) single-bucket resolution instead of listing + filtering.
- Removes an unnecessary transitive Storage-Proxy dependency from a control-plane read.
- The data is already in the DB (the list endpoints return it without any usage call); this is exposing it per-id.

## Citations (`6d680960d`)

- Usage-scan chain: `manager/services/vfolder/services/vfolder.py:502` `get()` → `:535` `get_proxy_and_volume` → `get_manager_facing_client(...)` → `:539` `get_folder_usage(...)`; tree walk on `storage/volumes/vfs/__init__.py`.
- DB-only list proof: `services/vfolder/services/vfolder.py:571` `list()` → `list_accessible_vfolders(...)` (no storage-proxy call).
- [docs/01 §4](../../01-architecture.md) rationale for avoiding `GET /folders/{id}`.

---
_Filed from the BAIFS project: see lablup/BAIFS#6 and `docs/spikes/upstream-asks/06-usage-free-metadata-endpoint.md` for full context and citations._

Contributor guide

Open the contributing guide

Research direction

Start with manager/services/vfolder/services/vfolder.py around get() at lines 502-539 and list() around line 571, then read docs/spikes/03-bucket-resolver.md. Compare the existing per-id and DB-only list paths; done means a per-id metadata response exposes the requested fields without invoking Storage-Proxy usage scanning, with coverage for the selected API behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.