[Feature] change InternLM2 modeling to unified type
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 748
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 54
Description
### Motivation
when do the w8a8 quantization in pytorch engine, I found that InternLM2 modeling like. It use self.attention, self.feed_forward...
```python
class InternLM2DecoderLayer(nn.Module):
def __init__(self, config: InternLM2Config):
super().__init__()
self.hidden_size = config.hidden_size
self.attention = INTERNLM2_ATTENTION_CLASSES[config.attn_implementation](config=config)
self.feed_forward = InternLM2MLP(config)
self.attention_norm = InternLM2RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
self.ffn_norm = InternLM2RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
```
but others model modeling like, and all use self.self_attn, self.mlp, they have the diff name.
```python
class LlamaDecoderLayer(nn.Module):
"""Decoder layer for Llama Model."""
def __init__(self, config: LlamaConfig):
super().__init__()
self.hidden_size = config.hidden_size
self.self_attn = LlamaAttention(config=config)
self.mlp = LlamaMLP(config)
self.input_layernorm = LlamaRMSNorm(config.hidden_size,
eps=config.rms_norm_eps)
self.post_attention_layernorm = LlamaRMSNorm(config.hidden_size,
eps=config.rms_norm_eps)
```
using unifiy name is important for quantization, and for other feature like Medusa, or Sparse Gemm(they need to change self.mlp layer).
**If we do not unify the names, we would need to continuously iterate through class names to find the layers that need to be quantized or replaced. This makes the code difficult to understand and also results in additional time consumption.**
> and if necessary, I will fix it later...
### Related resources
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.