No test coverage for image generation metrics and `image-gen` extra
- Dominant language
- Python
- Stars
- 12
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
## Description
`pyproject.toml` defines an `image-gen` optional dependency group with heavy dependencies:
```toml
image-gen = [
"Pillow>=10.0",
"transformers>=4.30",
"torch>=2.0",
]
```
However:
- There are no tests specifically covering image generation metrics (`clip_score`, `aesthetic_score`, `image_quality`, `image_faithfulness`, etc.)
- CI runs `--all-extras` which would pull in torch/transformers but doesn't validate them
- The multimodal metrics in `fasteval/metrics/multimodal.py` and `fasteval/metrics/vision.py` have no dedicated test files
## Suggested Fix
- Add test files for vision and image generation metrics (can use mocks to avoid requiring GPU/large model downloads in CI)
- Consider whether `--all-extras` in CI should exclude `image-gen` to keep CI fast, with a separate job for multimodal tests
Contributor guide
Assessment
This issue has not been assessed yet.