NVIDIA-NeMo / NVIDIA-NeMo/Curator

[26.04 POR] Benchmarking for Audio: stage-wise profiling across modalities

Open
#1,651 0 comments 0 reactions 1 assignee View on GitHub

@mohammadaaftabv is already working on this.

Since Mar 24, 2026.

Dominant language
Python
Stars
1.8k
Forks
327
Avg merge
4d 5h
Merged PRs (30d)
30

Description

Goal: Stage-wise profiling across modalities (Text / Audio / Image / Video / Reasoning) to identify bottlenecks and prioritize optimizations.

Acceptance (draft)

Define profiling methodology + metrics per stage.
Produce baseline results per modality.
Summarize top bottlenecks + proposed fixes.

Checklist:
Release Readiness Checklist
Benchmarking and Performance

Security and Dependencies

  • Review vulnerability scans for CVEs.
  • If dependency updates are required for a modality (due to CVEs or breakage), the modality PIC must help resolve and unblock the release.
  • Validate that all open-source packages comply with licensing policies.
  • For any packages or binaries with unknown or unclear licensing, the modality PIC must coordinate with Legal to verify dependencies and ensure release readiness.
  • Validate that all NVIDIA internal packages included in the container are authorized and correctly versioned.

Quality and Documentation

  • Identify and address all release-blocking bugs.
  • Update all relevant documentation and release notes.

Visibility and Communication

  • Maintain clear visibility of work status to stakeholders and the project team.

Ref: https://nvidia.atlassian.net/wiki/spaces/NLLMS/pages/3142129441/NeMo+Curator+Release+Readiness#Modality-PIC-Responsibilities

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.