ContextLab / ContextLab/orchestrator
Bring the legacy test suite to green (~540 failures outside the blocking layer)
- Dominant language
- Python
- Stars
- 3
- Forks
- 2
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
## Overview
Following the major refactor in #307, we need to systematically audit our entire test suite to achieve a 100% pass rate while using tests to guide development.
## Current Status
- 108 test files have been updated with correct import paths (orchestrator. → src.orchestrator.)
- Many tests may be obsolete due to architectural changes in refactor #307
- Some tests are broken but should work with current architecture
- Failing tests should guide toolbox improvements and missing functionality
## Systematic Approach
### Phase 1: Test Categorization
- [ ] Run complete test suite and capture all failures
- [ ] Categorize each failing test as:
- **OBSOLETE**: No longer relevant after refactor, should be removed
- **BROKEN**: Should work with current architecture, needs repair
- **GUIDE**: Reveals missing functionality that needs implementation
### Phase 2: Test Cleanup
- [ ] Remove all OBSOLETE tests that no longer apply to current architecture
- [ ] Document removed tests and reasoning
### Phase 3: Test Repair
- [ ] Fix all BROKEN tests to work with current architecture
- [ ] Update test assertions to match new API contracts
- [ ] Ensure tests use correct import paths and class structures
### Phase 4: Test-Driven Development
- [ ] Use GUIDE tests to implement missing functionality
- [ ] Create new GitHub issues for large-scale implementations
- [ ] Fix small missing pieces immediately
### Phase 5: Validation
- [ ] Achieve 100% test pass rate locally
- [ ] Verify CI passes across all platforms (Ubuntu, macOS, Windows)
- [ ] Verify CI passes across all Python versions (3.11, 3.12)
## Success Criteria
- 100% test pass rate locally and in CI
- All obsolete tests removed and documented
- All broken tests repaired or converted to implementation guides
- Test suite guides future development effectively
## Related Issues
- Closes: orchestrator-api-completion epic
- Related: #307 (major refactor)
- Builds on: Import path fixes across 108 test files
Contributor guide
Assessment
This issue has not been assessed yet.