OpenAI api seems to not be working
- 主要語言
- Rust
- 星號
- 619
- 分支
- 74
- PR 合併指標
- 30 天內沒有已合併 PR
描述
I'm trying to run this script, but I keep getting nonsense output from the openai api
```
import requests
import json
from typing import Optional
# Function to call the OpenAI API with optional logit_bias
def test_logit_bias(api_key: str, prompt: str, max_tokens: int, logit_bias: Optional[dict] = None):
# Endpoint URL for OpenAI API
openai_api_base = "http://0.0.0.0:65530/api/oai"
api_url = f"{openai_api_base}/v1/completions"
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {api_key}'
}
# Payload with and without logit_bias
payload = {
"prompt": prompt,
"max_tokens": 3,
"model": "rwkv",
"temperature": 0
}
if logit_bias:
payload["logit_bias"] = logit_bias
response = requests.post(api_url, headers=headers, json=payload)
if response.status_code == 200:
print(json.dumps(response.json(), indent=2))
else:
print(f"Error {response.status_code}: {response.text}")
# Usage
API_KEY = "your-api-key" # Replace with your actual OpenAI API key
# Test with logit_bias
test_logit_bias(
api_key=API_KEY,
prompt="A profression that involves going to space is an ",
max_tokens=100,
logit_bias={15311: -100} # Replace with actual token values
)
# Test without logit_bias
print("Without")
test_logit_bias(
api_key=API_KEY,
prompt="A profression that involves going to space is an ",
max_tokens=100
)
```
This is the response I get.
```
{
"object": "text_completion",
"model": "assets/models/rwkv-4.safetensors",
"choices": [
{
"text": "\ud83d\ude80",
"index": 0,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 3,
"total_tokens": 14
}
}
Without
{
"object": "text_completion",
"model": "assets/models/rwkv-4.safetensors",
"choices": [
{
"text": "\ud83d\ude80",
"index": 0,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 3,
"total_tokens": 14
}
}
```
The weird thing is, i'm able to use the chat page in the UI just fine.
貢獻指南
這個儲存庫沒有索引到貢獻指南
評估
這個 Issue 還沒有評估資料。