abetlen / abetlen/llama-cpp-python

Wrong `chat_format` documented for Moondream 2

Abierto
#1,996 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Hi! 👋

I was testing the Server API with Moondream 2 as the model, and the [documented `chat_format`](https://github.com/abetlen/llama-cpp-python/blob/37eb5f0a4c2a8706b89ead1406b1577c4602cdec/README.md#L504), `moondream2`, threw an error:

```plain
Error code: 500 - {'error': {'message': "Invalid chat handler: moondream2 (valid formats: ['llama-2', 'llama-3', 'alpaca', 'qwen', 'vicuna', 'oasst_llama', 'baichuan-2', 'baichuan', 'openbuddy', 'redpajama-incite', 'snoozy', 'phind', 'intel', 'open-orca', 'mistrallite', 'zephyr', 'pygmalion', 'chatml', 'mistral-instruct', 'chatglm3', 'openchat', 'saiga', 'gemma', 'functionary', 'functionary-v2', 'functionary-v1', 'chatml-function-calling'])", 'type': 'internal_server_error', 'param': None, 'code': None}}
```

[Looking at the code](https://github.com/abetlen/llama-cpp-python/blob/37eb5f0a4c2a8706b89ead1406b1577c4602cdec/llama_cpp/server/model.py#L118), it appears that the correct `chat_format` is `moondream` and not `moondream2` (`moondream` worked as expected).

What do you think?

Let me know if I can open a PR to fix the documented `chat_format`, please.

Thank you for this project! 😄

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.