abetlen / abetlen/llama-cpp-python

Thinking toggle support for Qwen related models

Abierto
#2,063 10 comentarios 3 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

**Is your feature request related to a problem? Please describe.**
Cannot toggle thinking in Qwen models, when we do it through the user prompt way, it still gives out opening and closing think tags.

**Describe the solution you'd like**
Support for params/ flags like reasoning_budget, chat_template_kwargs parameters.
**Describe alternatives you've considered**
N/A

**Additional context**
This problem has been a long point of discussion in the upstream project (llama.cpp) too and I am hoping for it to trickle down here and have support for it, noting that the PRs have been merged and llama.cpp has controls over thinking now.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.