bigscience-workshop / bigscience-workshop/Megatron-DeepSpeed

"Mask is silently ignored due to the use of a custom kernel" with pretrain_gpt_single_node.sh

Open
#320 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
226
PR merge metrics
No merged PRs in 30d

Description

## Update

The issue turned out to be DeepSpeed usages of `pretrain_gpt_single_node.sh`.
I will make a pull request soon.

## Original Report

Please let me know what details I shall provide, thank you!

Python 3.7.12
pt=1.11.0+cu113, cuda=11.3

Using `pretrain_gpt_single_node.sh` with the instructions on README:
```
[...]/megatron/model/fused_softmax.py", line 191, in forward_fused_softmax
assert mask is None, "Mask is silently ignored due to the use of a custom kernel"
```

Also tried `--no-masked-softmax-fusion` but no differences except the line.
```
[...]/megatron/model/fused_softmax.py", line 218, in forward_torch_softmax
assert mask is None
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.