Add Support for Free-form Chat
- Vorherrschende Sprache
- Python
- Sterne
- 0
- Forks
- 1
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
Currently, the `/v1/chat/completions` endpoint only supports structured output and returns an error if response_format is not provided. We should add support for standard, free-form text generation to make the endpoint fully OpenAI-compatible.
**Acceptance Criteria**
- When a request is made to /v1/chat/completions without a response_format object, the server should generate a normal text response.
- When a request is made with a response_format object, it should continue to work as it does now.
**Implementation Guide**
- This is a great first issue for someone familiar with FastAPI.
- Add a generate_freeform method to artisan_engine/adapter.py: This new method will call the underlying _llama_model directly to get a standard text completion.
```python
# in LlamaCppAdapter
def generate_freeform(self, prompt: str, **kwargs) -> str:
if not self.is_loaded():
self.load_model()
response = self._llama_model(prompt, **kwargs)
return response["choices"][0]["text"]
```
Update the else block in the chat_completions endpoint in main.py: Instead of raising an error, it should now call the new adapter.generate_freeform method.
```python
# in main.py, inside chat_completions
else:
# No structured output requested, perform free-form generation.
result = adapter.generate_freeform(
prompt=prompt,
max_tokens=request.max_tokens or 256,
temperature=request.temperature or 1.0,
# Pass other valid params
)
```
Add a unit test to tests/test_api.py that calls the endpoint without response_format and asserts that it gets a 200 OK response.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.