lablup / lablup/backend.ai

Scenario-based test tree (tests/scenario) with a domain-scoped test kit

Open
#14,437 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

Component tests today cost ~46% of their line count in conftest setup (54 per-domain route-registry rebuilds, a 1,625-line root conftest, hand-listed FK-ordered table lists), and their import closure is far wider than what they exercise: a domain component test pulls 2,653 of 3,941 manager files, so CI cannot use transitive test selection without running everything.

This epic introduces a new test tree, tests/scenario/, where a test is a table of scenarios (actor / given / when / then) run through the transport-agnostic adapter layer (Pydantic \*Input -> \*Payload) against a real database, with a small kit that only assembles src code (no row-building logic in tests). The tree has its own lean root conftest, per-domain wiring modules, and Pants dependency rules so that a scenario file's closure equals the domain's real chain (~330 files) plus a shared core; CI can then select tests transitively for this tree.

Phase 1 (this epic now): two foundation refactors that hold regardless of the scenario design, plus one PoC that tries a simple and a complex scenario end to end and reports the open design decisions. Phase 2 (after the PoC review): kit, tree skeleton, domain pilot, CI job, conventions. Phase 3 (separate issues): migrate domains in batches.

Design notes: hejo drafts/test-framework/issues.md (기반/규약/적용 sections, 시나리오 전환 설계, 컴포넌트 테스트 구성).

JIRA Issue: BA-7776

Contributor guide

Open the contributing guide

Research direction

Start with hejo drafts/test-framework/issues.md, especially the foundation, conventions, scenario-transition design, and component-test sections. Then identify the two Phase 1 foundation refactors and build a PoC in tests/scenario/ covering one simple and one complex scenario through the adapter layer against a real database. Done means the PoC documents open design decisions and provides enough evidence for the Phase 2 design.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, databases, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.