lablup / lablup/backend.ai

Validate fixture values against application enums during `mgr fixture populate`

Open
#11,637 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Background

`mgr fixture populate` (`src/ai/backend/manager/cli/fixture.py` → `manager/models/base.py::populate_fixture`) inserts rows directly into tables after coercing only SQL-level types: `DateTime`, SQLAlchemy `EnumType` / `StrEnumType` / `EnumValueType`, and structured JSON columns. Columns declared as plain `sa.String` but logically backed by a Python enum (e.g. `permissions.scope_type` ↔ `RBACElementType`, `permissions.entity_type` ↔ `RBACElementType`, `permissions.operation` ↔ `OperationType`) are inserted verbatim without any value validation.

This let BA-6059 happen: fixtures shipped 11 rows with `scope_type:"global"` even though `RBACElementType` no longer has `GLOBAL`. The bad value passed through `populate_fixture` silently and only blew up later inside the GraphQL adapter (`_permission_data_to_node` → `RBACTypeConversionError`).

## Goal

Catch this class of error at fixture-load time with a clear `"table.column = 'global' is not a valid RBACElementType"` message, instead of letting it surface as a runtime GraphQL exception far downstream.

## Scope of work

- Add a fixture-time validator that, for each row, checks string columns whose value is expected to belong to a known application enum.
- Provide a way to declare the expected enum per `(table, column)`. Options:
- a registry mapping (e.g. in `manager/models/base.py` or a dedicated `fixture_validation.py`) that lists `(table_name, column_name) → enum_cls`, OR
- per-Row class annotation/metadata that the validator can reflect on.
The simpler registry approach is acceptable as a first cut.
- Cover at least the RBAC-related columns that motivated this issue: `permissions.scope_type`, `permissions.entity_type`, `permissions.operation`, `association_scopes_entities.scope_type`, `association_scopes_entities.entity_type`.
- On validation failure, raise `DataTransformationFailed` with row index + column + offending value + expected enum class. Fail fast: do not insert any row from the fixture if any row fails.

## Out of scope

- Backfilling exhaustive validation across all tables — the registry can grow incrementally.
- JSON-schema-style structural validation of the fixture file.

## Success Criteria

- [ ] A fixture row with `scope_type:"global"` for the `permissions` table causes `mgr fixture populate` to exit non-zero before any insert, with a message naming the table, column, value, and expected enum.
- [ ] All currently-shipped fixtures (`fixtures/manager/**.json`, `src/ai/backend/install/fixtures/**.json`) pass the validator unchanged after the BA-6059 cleanup.
- [ ] Validator does not regress legitimate enum values: `scope_type:"domain"`, `scope_type:"user"`, `scope_type:"project"` continue to insert successfully.
- [ ] Unit test covers: (a) valid value passes, (b) invalid enum value raises `DataTransformationFailed`, (c) column not in registry is left untouched (no false positive).
- [ ] pants test passes for affected packages.

JIRA Issue: BA-6060

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.