aafre / aafre/artisan-engine

Add Support for Free-form Chat

Aperta
#7 0 commenti 0 reazioni 1 assegnatario Rivendicata da @aafre Vedi su GitHub
api enhancement Python
Lingua principale
Python
Stelle
0
Fork
1
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Currently, the `/v1/chat/completions` endpoint only supports structured output and returns an error if response_format is not provided. We should add support for standard, free-form text generation to make the endpoint fully OpenAI-compatible.

**Acceptance Criteria**
- When a request is made to /v1/chat/completions without a response_format object, the server should generate a normal text response.
- When a request is made with a response_format object, it should continue to work as it does now.

**Implementation Guide**
- This is a great first issue for someone familiar with FastAPI.
- Add a generate_freeform method to artisan_engine/adapter.py: This new method will call the underlying _llama_model directly to get a standard text completion.

```python
# in LlamaCppAdapter
def generate_freeform(self, prompt: str, **kwargs) -> str:
if not self.is_loaded():
self.load_model()
response = self._llama_model(prompt, **kwargs)
return response["choices"][0]["text"]
```
Update the else block in the chat_completions endpoint in main.py: Instead of raising an error, it should now call the new adapter.generate_freeform method.

```python
# in main.py, inside chat_completions
else:
# No structured output requested, perform free-form generation.
result = adapter.generate_freeform(
prompt=prompt,
max_tokens=request.max_tokens or 256,
temperature=request.temperature or 1.0,
# Pass other valid params
)
```

Add a unit test to tests/test_api.py that calls the endpoint without response_format and asserts that it gets a 200 OK response.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.