BOHICA-LABS / BOHICA-LABS/writescore
bug: Dimension weight test expectations don't match actual scoring behavior
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
Four unit tests in `test_dual_score_calculator.py` fail because their hardcoded expected values don't match the actual dimension weights used by the scoring system. The tests assert specific score values based on assumed weights (18.1 for predictability, 12.8 for advanced_lexical), but the actual calculated scores differ by ~2.5-3%.
## Severity
**Medium** - Tests fail but core functionality works correctly. The scoring system produces valid scores; only the test expectations are incorrect.
## Failing Tests
**File:** `tests/unit/scoring/test_dual_score_calculator.py`
| Test | Expected | Actual | Delta |
|------|----------|--------|-------|
| `test_gltr_dimension_high` | 17.195 | 17.68 | +2.8% |
| `test_gltr_dimension_low` | 4.525 | 4.65 | +2.8% |
| `test_mattr_dimension_excellent` | 12.16 | 12.51 | +2.9% |
| `test_mattr_dimension_poor` | 3.2 | 3.29 | +2.8% |
## Root Cause
The tests calculate expected scores using:
```
expected_score = (dimension_score / 100.0) * dimension_weight
```
Comments in the test file reference Story 2.4.1 weight rescaling (109% → 100%), but the expected values don't match either the original or rescaled weights.
Likely cause: When `figurative_language` dimension fails to load (missing `sentence-transformers`), its weight may be redistributed to other dimensions, causing the ~2.8% increase.
## Steps to Reproduce
```bash
uv run pytest tests/unit/scoring/test_dual_score_calculator.py::TestCalculateDualScoreDimensions::test_gltr_dimension_high -v
```
Output:
```
E assert 17.679499999999997 == 17.195 ± 0.01
E Obtained: 17.679499999999997
E Expected: 17.195 ± 0.01
```
## Proposed Fix
1. Add debug logging to understand runtime weight normalization
2. Document the normalization behavior in architecture docs
3. Update test expectations to match actual behavior with explanatory comments
4. Consider adding a test for the weight normalization logic itself
## Discovery Context
Discovered during Epic 6: Developer Experience Improvements. Verified to fail identically on `main` branch before any Epic 6 changes - this is a pre-existing issue.
## Related Files
- `tests/unit/scoring/test_dual_score_calculator.py` (lines 558-622)
- `src/writescore/scoring/dual_score_calculator.py`
- `src/writescore/dimensions/predictability.py` (weight = 18.1)
- `src/writescore/dimensions/advanced_lexical.py` (weight = 12.8)
## Labels
bug, testing, good first issue
Contributor guide
Assessment
This issue has not been assessed yet.