[Bug] Deepseek-V2 MOE 模型加载适配问题

### Checklist

- [x] 1. I have searched related issues but cannot get the expected help.
- [x] 2. The bug has not been fixed in the latest version.
- [x] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

### Describe the bug

您好，我尝试着适配 deepseekv2 的 AWQ 版本，在适配的过程中发现主要是由于 deepseek-v2 模型在第一层是传统的 ffn 而在后面是 moe 层，在 transformers block 加载的时候会加载 NoneType 因此我做了判断
```python
def __call__(self, i: int, r: BaseReader):
# print(self.model.model_config.expert_num)
if i >= 0:
print("expert num: ", self.model.model_config.expert_num[i])
for m in self.modules:
if self.model.model_config.expert_num[i] == 0:
if m.__class__ is MoeFfn:
print('moe ffn omit: ', i, m)
continue
else:
if m.__class__ is Ffn:
print('ffn omit: ', i, m)
continue
m(i, r)
return 1
else:
self.misc(i, r)
```
虽然之后可以正常 convert to turbomind model，但是会出现 warning
```bash
WARNING - turbomind.py:221 - the model may not be loaded successfully with 234 uninitialized params:
['layers.22.feed_forward.w1.0.qweight', 'layers.3.feed_forward.w1.0.qweight', 'layers.5.feed_forward.w1.0.qweight', 'layers.12.feed_forward.w2.0.qweight', 'layers.16.feed_forward.w1.0.scales', 'layers.7.feed_forward.w3.0.zeros', 'layers.26.feed_forward.w3.0.zeros', 'layers.1.feed_forward.w3.0.scales', 'layers.18.feed_forward.w2.0.zeros', 'layers.3.feed_forward.w1.0.scales', 'layers.18.feed_forward.w1.0.zeros', 'layers.14.feed_forward.w2.0.scales', 'layers.8.feed_forward.w1.0.qweight', 'layers.3.feed_forward.w3.0.qweight', 'layers.3.feed_forward.w2.0.scales', 'layers.24.feed_forward.w2.0.qweight', 'layers.9.feed_forward.w1.0.zeros', 'layers.12.feed_forward.w1.0.qweight', 'layers.1.feed_forward.w2.0.qweight', 'layers.17.feed_forward.w3.0.zeros',
```

然后 turbomind 会报错
[TM][ERROR] void turbomind::LlamaLinear::Impl::forwardInt4(T*, turbomind::LlamaLinear::Pitched, int, const turbomind::LlamaDenseWeight&, turbomind::LlamaLinear::Type) [with T = __half]: -1

8192 8192 0 4096 512 0 0 0 0
感觉逻辑上是虽然没有convert ffn 层了但是 cpp 的部分还是会在每层都去加载 ffn 层从而有一些权重没有找到而报错，想问一下有没有一些可以避免加载的方法，或者有什么可以暂时的方法支持一些 deepseek-v2 / v3 awq 呢

### Reproduction

lmdeploy serve api_server /workspace/models/deepseek-v2-awq --server-name 0.0.0.0 --server-port 2333 --tp 1 --backend turbomind --chat-template deepseek

### Environment

```Shell
sys.platform: linux
Python: 3.10.12 (main, Feb 4 2025, 14:57:36) [GCC 11.4.0]
CUDA available: True
MUSA available: False
numpy_random_seed: 2147483648
GPU 0,1,2,3,4,5,6,7: NVIDIA H100 80GB HBM3
CUDA_HOME: /usr/local/cuda
NVCC: Cuda compilation tools, release 12.4, V12.4.131
GCC: x86_64-linux-gnu-gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
PyTorch: 2.5.1+cu121
PyTorch compiling details: PyTorch built with:
- GCC 9.3
- C++ Version: 201703
- Intel(R) oneAPI Math Kernel Library Version 2024.2-Product Build 20240605 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v3.5.3 (Git Hash 66f0cb9eb66affd2da3bf5f8d897376f04aae6af)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX512
- CUDA Runtime 12.1
- NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
- CuDNN 90.1 (built against CUDA 12.4)
- Magma 2.6.1
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=9.1.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DLIBKINETO_NOXPUPTI=ON -DUSE_FBGEMM -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, TORCH_VERSION=2.5.1, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

TorchVision: 0.20.1+cu121
LMDeploy: 0.7.1+c4d5bd9
transformers: 4.47.1
gradio: 5.19.0
fastapi: 0.115.8
pydantic: 2.10.6
triton: 3.1.0
NVIDIA Topology:
GPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 NIC0 NIC1 NIC2 NIC3 NIC4 CPU Affinity NUMA Affinity GPU NUMA ID
GPU0 X NV18 NV18 NV18 NV18 NV18 NV18 NV18 NODE NODE SYS SYS SYS 0,2,4,6,8,10 0 N/A
GPU1 NV18 X NV18 NV18 NV18 NV18 NV18 NV18 PIX NODE SYS SYS SYS 0,2,4,6,8,10 0 N/A
GPU2 NV18 NV18 X NV18 NV18 NV18 NV18 NV18 NODE PIX SYS SYS SYS 0,2,4,6,8,10 0 N/A
GPU3 NV18 NV18 NV18 X NV18 NV18 NV18 NV18 NODE NODE SYS SYS SYS 0,2,4,6,8,10 0 N/A
GPU4 NV18 NV18 NV18 NV18 X NV18 NV18 NV18 SYS SYS NODE NODE PIX 1,3,5,7,9,11 1 N/A
GPU5 NV18 NV18 NV18 NV18 NV18 X NV18 NV18 SYS SYS PIX NODE NODE 1,3,5,7,9,11 1 N/A
GPU6 NV18 NV18 NV18 NV18 NV18 NV18 X NV18 SYS SYS NODE NODE NODE 1,3,5,7,9,11 1 N/A
GPU7 NV18 NV18 NV18 NV18 NV18 NV18 NV18 X SYS SYS NODE PIX NODE 1,3,5,7,9,11 1 N/A
NIC0 NODE PIX NODE NODE SYS SYS SYS SYS X NODE SYS SYS SYS
NIC1 NODE NODE PIX NODE SYS SYS SYS SYS NODE X SYS SYS SYS
NIC2 SYS SYS SYS SYS NODE PIX NODE NODE SYS SYS X NODE NODE
NIC3 SYS SYS SYS SYS NODE NODE NODE PIX SYS SYS NODE X NODE
NIC4 SYS SYS SYS SYS PIX NODE NODE NODE SYS SYS NODE NODE X

Legend:

X = Self
SYS = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
PHB = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
PXB = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
PIX = Connection traversing at most a single PCIe bridge
NV# = Connection traversing a bonded set of # NVLinks

NIC Legend:

NIC0: mlx5_0
NIC1: mlx5_1
NIC2: mlx5_4
NIC3: mlx5_5
NIC4: mlx5_bond_0
```

### Error traceback

```Shell

```