NVIDIA-NeMo / NVIDIA-NeMo/Curator
[Docs] Document video caption-quality evaluation with Summarize-then-Align
Open
@lbliii is already working on this.
Since Jun 30, 2026.
documentation
tutorials
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 327
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 30
Description
Context
Child of #2118. #1980 added eval/video/ tooling for caption-quality evaluation with CosmosEmbed1. Documentation exists only in eval/video/README.md, outside the published Fern site.
Scope
- Benchmark-dataset construction: sampling, embedding, K-means selection, and expected artifacts
- Caption scoring: summarization, CosmosEmbed1 encoding, cosine similarity, caching, and CSV outputs
- Model/data prerequisites and expected directory layouts
- Interpretation and limitations of the baseline scores
Acceptance criteria
- A published Fern evaluation guide is added and linked from video captioning docs
- Both scripts have tested commands and complete argument explanations
- Input/output schemas and directory layouts are shown
- Hardware/model-download expectations and reproducibility limits are explicit
- The distinction between evaluation tooling and the main curation pipeline is clear
- Fern checks pass
Related PR: #1980
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.