aafre / aafre/artisan-engine

Add Support for Free-form Chat

Abierto
#7 0 comentarios 0 reacciones 1 asignado Reclamado por @aafre Ver en GitHub
api enhancement Python
Lenguaje dominante
Python
Estrellas
0
Forks
1
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Currently, the `/v1/chat/completions` endpoint only supports structured output and returns an error if response_format is not provided. We should add support for standard, free-form text generation to make the endpoint fully OpenAI-compatible.

**Acceptance Criteria**
- When a request is made to /v1/chat/completions without a response_format object, the server should generate a normal text response.
- When a request is made with a response_format object, it should continue to work as it does now.

**Implementation Guide**
- This is a great first issue for someone familiar with FastAPI.
- Add a generate_freeform method to artisan_engine/adapter.py: This new method will call the underlying _llama_model directly to get a standard text completion.

```python
# in LlamaCppAdapter
def generate_freeform(self, prompt: str, **kwargs) -> str:
if not self.is_loaded():
self.load_model()
response = self._llama_model(prompt, **kwargs)
return response["choices"][0]["text"]
```
Update the else block in the chat_completions endpoint in main.py: Instead of raising an error, it should now call the new adapter.generate_freeform method.

```python
# in main.py, inside chat_completions
else:
# No structured output requested, perform free-form generation.
result = adapter.generate_freeform(
prompt=prompt,
max_tokens=request.max_tokens or 256,
temperature=request.temperature or 1.0,
# Pass other valid params
)
```

Add a unit test to tests/test_api.py that calls the endpoint without response_format and asserts that it gets a 200 OK response.

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.