lablup / lablup/backend.ai

Reduce test re-run scope by fixing layer violations and splitting Pants targets

Open
#12,327 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

## Problem

Running `pants test --changed-dependents=transitive` re-runs 826 tests even when only a few files change. Two root causes:

1. 12 layer-violating imports — lower layers (repositories) import upper layers (services, api), creating dependency cycles.
2. Single Pants target — the entire manager module (1,596 modules) is one target, so changing a single file invalidates the whole target.

Declared layer order: API -> Processor -> Service -> Repository -> DB

## Action Items

### Step 1: Fix layer violations (code structure)

Remove imports that violate the layer order and move the affected types to their correct location:

- UserCreateSpec, UserUpdateSpec -> repositories/user/types.py
- StringFilter, IntFilter, BaseFilterAdapter -> data/filter/
- EndpointStatistics, KernelStatistics -> data/deployment/statistics.py
- Undefined sentinel -> data/common/sentinel.py
- model_serving exceptions -> data/model_serving/exceptions.py

### Step 2: Split Pants targets (build system optimization)

After Step 1, add per-subpackage BUILD files:

- manager/data/BUILD
- manager/models/BUILD
- manager/repositories/BUILD
- manager/services/BUILD

Effect: changing data/deployment/types.py reduces re-run scope from 826 to ~100 tests.

## Success Criteria
- [ ] all 12 layer-violating imports removed; declared layer order (api -> services -> repositories -> models) holds with no upward imports
- [ ] moved types (UserCreateSpec/UserUpdateSpec, StringFilter/IntFilter/BaseFilterAdapter, EndpointStatistics/KernelStatistics, Undefined sentinel, model_serving exceptions) resolve from their new locations; no stale imports remain
- [ ] per-subpackage BUILD files added for manager/data, manager/models, manager/repositories, manager/services
- [ ] editing manager/data/deployment/types.py and running pants test --changed-dependents=transitive re-runs ~100 tests (down from 826)
- [ ] pants lint/check passes for affected packages
- [ ] pants test passes for affected packages

JIRA Issue: BA-6573

Contributor guide

Open the contributing guide

Research direction

Start by inspecting the manager/data, manager/models, manager/repositories, and manager/services packages, tracing the 12 layer-violating imports and the types listed for relocation. Add the four per-subpackage BUILD files after the moves, then run Pants lint/check and tests for affected packages. Done means no stale imports remain and changing manager/data/deployment/types.py reruns about 100 rather than 826 tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, build-system
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.