NVIDIA / NVIDIA/Megatron-LM

[QUESTION]How to set the routed scaling factor

Open
#1,504 2 comments 0 reactions 0 assignees View on GitHub
community-request question
Dominant language
Python
Stars
17.9k
Forks
4.5k
Avg merge
4d 3h
Merged PRs (30d)
272

Description

How is the routed scaling factor determined, by experiment or by calculation?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.