If create the Linear() casually, it won't be trained during training.

https://github.com/graykode/nlp-tutorial/blob/3b3a80dc63e69935731bcf09c951eb371692af8f/5-1.Transformer/Transformer(Greedy_decoder)-Torch.py#L98

Start at 5-1.Transformer/Transformer(Greedy_decoder)-Torch.py around line 98 and trace how the Linear layer is created and reached during training. Check the model's parameter registration and optimizer path; done means the issue's concern about the layer not being trained is resolved and the relevant training behavior is verified.

Written by the indexing model from the issue text.