NVIDIA-NeMo / NVIDIA-NeMo/Curator
[26.04 POR] Benchmarking for Audio: stage-wise profiling across modalities
Open
@mohammadaaftabv is already working on this.
Since Mar 24, 2026.
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 327
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 30
Description
Goal: Stage-wise profiling across modalities (Text / Audio / Image / Video / Reasoning) to identify bottlenecks and prioritize optimizations.
Acceptance (draft)
Define profiling methodology + metrics per stage.
Produce baseline results per modality.
Summarize top bottlenecks + proposed fixes.
Checklist:
Release Readiness Checklist
Benchmarking and Performance
- Add benchmarking scripts to the repository and integrate them into the regular CI matrix (https://github.com/NVIDIA-NeMo/Curator/issues/1563).
- Run benchmarks for all modality pipelines and capture performance results.
Security and Dependencies
- Review vulnerability scans for CVEs.
- If dependency updates are required for a modality (due to CVEs or breakage), the modality PIC must help resolve and unblock the release.
- Validate that all open-source packages comply with licensing policies.
- For any packages or binaries with unknown or unclear licensing, the modality PIC must coordinate with Legal to verify dependencies and ensure release readiness.
- Validate that all NVIDIA internal packages included in the container are authorized and correctly versioned.
Quality and Documentation
- Identify and address all release-blocking bugs.
- Update all relevant documentation and release notes.
Visibility and Communication
- Maintain clear visibility of work status to stakeholders and the project team.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.