[Bug] AttributeError: 'LlavaNextConfig' object has no attribute 'hidden_size'

### Checklist

- [X] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.

### Describe the bug

download the llava-v1.6-vicuna-7b-hf model to local folder, and run the command:
lmdeploy serve api_server /data/models/huggingface/hub/llava-v1.6-vicuna-7b-hf

It reports error:
llava-1 | 2024-06-27 11:10:15,226 - lmdeploy - WARNING - Try to run with pytorch engine because `/data/models/huggingface/hub/llava-v1.6-vicuna-7b-hf` is not explicitly supported by lmdeploy.
llava-1 | 2024-06-27 11:10:22,420 - lmdeploy - ERROR - AttributeError: 'LlavaNextConfig' object has no attribute 'hidden_size'
llava-1 | 2024-06-27 11:10:22,421 - lmdeploy - ERROR - test failed!
llava-1 | ("Checking failed with error 'LlavaNextConfig' object has no attribute 'hidden_size'", 'Please send issue to LMDeploy with error logs.')

### Reproduction

download the llava-v1.6-vicuna-7b-hf model to local folder, and run the command:

lmdeploy serve api_server /data/models/huggingface/hub/llava-v1.6-vicuna-7b-hf

### Environment

```Shell
Use latest docker image run the command.
```

### Error traceback

_No response_