lablup / lablup/backend.ai

Route v2 scope-action reads through the virtual scope chain instead of legacy ownership columns

Open
#14,050 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

Permission checks and every ownership relation are settled on the virtual scope chain (`virtual_scopes` / `scope_bindings` / `entity_memberships`). BA-6656 covers the write side and permission resolution. This epic covers the read side: the search and list queries behind v2 scope actions, which still bound their rows with legacy foreign-key columns and legacy association tables.

## Survey

Full sweep of the 77 v2 scope-action classes under `ai.backend.manager.actions.v2.scope.BaseScopeAction` and the 48 `OperationScope` implementations under `src/ai/backend/manager/models/*/scopes.py`. Exactly 2 of the 48 read through the virtual scope chain today, both via `models/virtual_scope/queries.py`: `UserProjectOperationScope` and `ProjectUserOperationScope`.

Reads reachable only from a legacy API surface (`api/gql_legacy`, REST v1) are out of scope. `api/gql_legacy` imports no adapter, so no item listed below is legacy-driven; every one of them is reached from `api/gql/*_v2` or `api/rest/v2`.

## P0 - the same relation is read from two different tables

- `ScopedRoleOperationScope` (`models/rbac_models/scopes.py`) reads a scope's roles from `association_scopes_entities`, the table `models/virtual_scope/queries.py` declares off-limits for new membership reads. `repositories/ops/v2/entity_write.py` already reads the same relation from `entity_memberships` joined to `virtual_scopes`.
- `UserFairShareOperationScope` resolves project membership by joining `association_groups_users` (`repositories/fair_share/db_source/db_source.py`), while `ProjectUserOperationScope` already resolves it from the virtual scope chain. The two APIs can disagree on who is in a project.

## P1 - write side already in place, convertible now

Each of these entities declares `member_of()`, so its `entity_memberships` edge is written at creation.

- `DomainUserOperationScope` - `users.domain_name == :name` -> domain VS membership
- `DomainProjectOperationScope` - `groups.domain_name == (SELECT name FROM domains ...)` -> domain VS membership
- `ProjectVFolderOperationScope` - `vfolders.group == :project_id` -> project VS membership
- `UserVFolderOperationScope` - `vfolders.user == :uid OR vfolder_permissions` -> user VS membership plus capped grants for the shared folders
- `ProjectSessionOperationScope` - `sessions.group_id == :project_id`
- `ProjectDeploymentOperationScope` - `endpoints.project == :project_id`; wired through the processor, service and repository but not yet reachable from any API handler
- `ProjectModelCardOperationScope` - `model_cards.project == :project_id`
- `VFolderModelCardOperationScope` - `model_cards.vfolder == :vfolder_id`; needs a vfolder parent edge added to the model card creator, which today declares project only
- `ProjectFairShareOperationScope` - `groups.domain_name == :name`

## P2 - blocked on a missing write side

These entities have no virtual scope node or membership edge yet, so the read cannot move until the write does.

- `UserKeypairOperationScope` - `keypairs.user`
- `UserErrorLogOperationScope` - `error_logs.user`
- `MyLoginSessionOperationScope` - `login_sessions.user_id`
- `MyLoginHistoryOperationScope` - `login_history.user_id`
- `EntityInvitationInviteeScope` - invitee resolved through an email subquery on `users`
- `EntityInvitationInviterScope` - `entity_invitations.inviter_user_id`
- `KernelKernelHistoryOperationScope` - unreachable today; its own docstring states it is what the read should scope by once virtual scopes land

## Many-to-many associations to replace with scope bindings

`resource_group` declares `member_of() == ()` and is a top-level entity, so "this domain / project / keypair may schedule on this resource group" is a `scope_bindings` edge rather than a row filter.

- `DomainResourceGroupOperationScope` - `sgroups_for_domains`
- `ProjectResourceGroupOperationScope` - `sgroups_for_projects`
- `UserResourceGroupOperationScope` - `sgroups_for_keypairs` through `keypairs.user`

## Scope actions with no OperationScope yet

These duplicate the relations above directly in the repository layer; each needs its scope extracted into an `OperationScope` before it can be converted.

- `ListModelServiceAction`, `SearchServicesAction` - `EndpointRow.session_owner == :uid`
- `ListTaskTemplatesAction`, `ListClusterTemplatesAction` - `session_templates.user_uuid` plus a `group_id` join
- `ExportUsersByDomainCSVAction` - `association_groups_users` join in `repositories/export/reports/user.py`
- `ExportSessionsByProjectCSVAction`, `ExportMyKeypairsCSVAction`, `ExportMySessionsCSVAction`
- `SearchSessionsAction`, `SearchKernelsAction`
- `ListVFolderAction`, `SearchStorageHostPermissionsAction`
- `GetDomainResourceOverviewAction`, `GetProjectResourceOverviewAction`
- `repositories/error_log/db_source/db_source.py` also reads project membership from `association_groups_users`

## Out of scope

25 of the 48 stay as they are:

- Parent entity to child field rows, where the row is owned by its parent and RBAC is decided on the parent: model card requirements, deployment preset slots, role preset permissions, role permissions and object permissions, the six scheduling history scopes, entity labels, the two audit log scopes, and the entity invitation target scope.
- Already keyed on a generic `(scope_type, scope_id)` pair: the idle checker assignment scope and the three app config fragment scopes. Turning these into virtual scope nodes is a separate decision.
- Aggregate tables whose partition key is the scope itself: the three resource usage bucket scopes.
- Not ownership at all: `RoleUserOperationScope` (role assignment), the two resource policy scopes (name references), and `DomainFairShareOperationScope` (a literal true).
- Anything reachable only from a legacy API surface (`api/gql_legacy`, REST v1).

## Blockers

- Entities that already declare `member_of()`: domain, project, user, vfolder, session, session_group, endpoint, model_card, image, network, resource_group, entity_invitation (target only), app_config_fragment.
- Entities that do not, and which every P2 item depends on: keypair, error_log, login_session, login_history, session_template, kernel, idle_checker_assignment, and the invitee/inviter edges of entity_invitation.
- `repositories/base/rbac/entity_creator.py` still writes only `AssociationScopesEntitiesRow`, so it diverges from the three-table write in `repositories/ops/v2/write_base.py`. The idle checker db source and the role manager still take that path.

## Done when

- No `OperationScope` classified above as an ownership read filters on a legacy ownership column or association table.
- `association_scopes_entities` and `association_groups_users` have no remaining membership readers outside the legacy gql/rbac compatibility paths.
- Every converted read has a test that proves it returns the same rows as the column-based query on backfilled data.

JIRA Issue: BA-7523

Contributor guide

Open the contributing guide

Research direction

Start with models/virtual_scope/queries.py and the OperationScope implementations under src/ai/backend/manager/models/*/scopes.py, comparing existing virtual-scope reads with repositories/ops/v2/entity_write.py. Trace the listed v2 actions and repositories, then add matching read tests on backfilled data. Done means converted ownership reads no longer use legacy columns or association tables, with compatibility paths excluded.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, databases
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.