MegEngine / MegEngine/InferLLM
没有支持最新的llama.cpp的格式吗
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 752
- Forks
- 94
- PR merge metrics
- No merged PRs in 30d
Description
我使用llama.cpp量化过的alpaca7b_plus,看样子并没有支持
```
llama_model_load_internal: format = ggjt v3 (latest)
```
```
main: seed = 1686927440
Assert ' 0 ' failed at file : /mnt/e/pyCode/InferLLM/src/graph/llama.cpp
line 37 : virtual void inferllm::LlamaGraph::load(std::shared_ptr, inferllm::LlmParams&, std::shared_ptr),
extra message: unsupported model type.Aborted
```
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at graph/llama.cpp line 37, where loading the quantized alpaca7b_plus model asserts on the unsupported model type. Compare the logged ggjt v3 format with the model types handled by this loader, then reproduce the failure with the reported model. Done means the latest llama.cpp-quantized format loads without the unsupported-model assertion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100