Add Support for Free-form Chat
- Langage dominant
- Python
- Étoiles
- 0
- Forks
- 1
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
Currently, the `/v1/chat/completions` endpoint only supports structured output and returns an error if response_format is not provided. We should add support for standard, free-form text generation to make the endpoint fully OpenAI-compatible.
**Acceptance Criteria**
- When a request is made to /v1/chat/completions without a response_format object, the server should generate a normal text response.
- When a request is made with a response_format object, it should continue to work as it does now.
**Implementation Guide**
- This is a great first issue for someone familiar with FastAPI.
- Add a generate_freeform method to artisan_engine/adapter.py: This new method will call the underlying _llama_model directly to get a standard text completion.
```python
# in LlamaCppAdapter
def generate_freeform(self, prompt: str, **kwargs) -> str:
if not self.is_loaded():
self.load_model()
response = self._llama_model(prompt, **kwargs)
return response["choices"][0]["text"]
```
Update the else block in the chat_completions endpoint in main.py: Instead of raising an error, it should now call the new adapter.generate_freeform method.
```python
# in main.py, inside chat_completions
else:
# No structured output requested, perform free-form generation.
result = adapter.generate_freeform(
prompt=prompt,
max_tokens=request.max_tokens or 256,
temperature=request.temperature or 1.0,
# Pass other valid params
)
```
Add a unit test to tests/test_api.py that calls the endpoint without response_format and asserts that it gets a 200 OK response.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
The work is in artisan_engine/adapter.py to add a generate_freeform method and in main.py to update the chat_completions endpoint. Start by reading the existing adapter code and the chat_completions function. Run the existing tests in tests/test_api.py, then add a new test for the free-form case. 'Done' is when a request without response_format returns a 200 OK with a text response.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- fastapi, python
- Domaine
- api, backend
- Type d'issue
- Fonctionnalité
- Difficulté
- 2/5
- Temps estimé
- 1-3 heures
- Activité
- À l'abandon
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 70/100