NVIDIA / NVIDIA/Megatron-LM

Enable custom model flops calculation

Open
#3,552 0 comments 0 reactions 1 assignee Claimed by @gautham-kollu View on GitHub
enhancement
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 6h
Merged PRs (30d)
271

Description

**Is your feature request related to a problem? Please describe.**
Need to support a custom model flops calculation interface if the generic calculation is not accurate for some custom model. Currently, only way is to build your special cases into the generic calculation. See Megatron-Bridge [here](https://github.com/NVIDIA-NeMo/Megatron-Bridge/blob/main/src/megatron/bridge/training/utils/flop_utils.py).

**Describe the solution you'd like**
Some interface through the ModelConfig or ModelBuilder introduced in https://github.com/NVIDIA-NeMo/Megatron-Bridge/pull/2241 that provides user with all necessary config and state to calculate model flops for their model on their own. Ideally, all models supported in Megatron-Bridge can use this interface instead of adding complexity to flop_utils.py (linked above).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.