TransformerLayer implementation is overly complex and difficult to follow

https://github.com/NVIDIA/TransformerEngine/blob/5aa4823dcc4942cadc2862adebef917758b6dc05/transformer_engine/pytorch/transformer.py#L70

I understand that you want to implement a single TransformerLayer class to support various model architectures.

However, this design leads to an issue: the code inside the TransformerLayer class has become overly complex, with too many parameters and a large number of if-else statements. For beginners, it is really difficult to figure out how to configure the parameters to fit their own model structures.

I'm curious why you didn't design it like the Hugging Face Transformers framework, where one model corresponds to one file:
https://github.com/huggingface/transformers/tree/main/src/transformers/models

This kind of design is much more user-friendly, because users only need to focus on the code they care about.

Start by reading transformer_engine/pytorch/transformer.py around line 70, then compare its single TransformerLayer design with the linked Hugging Face model layout. The issue does not define a specific decomposition, acceptance criteria, or tests, so the desired refactor and definition of done would need agreement before implementation.