Ai00-X / Ai00-X/ai00_server

OpenAI api seems to not be working

Aperta
#47 6 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Rust
Stelle
619
Fork
74
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

I'm trying to run this script, but I keep getting nonsense output from the openai api

```
import requests
import json
from typing import Optional

# Function to call the OpenAI API with optional logit_bias
def test_logit_bias(api_key: str, prompt: str, max_tokens: int, logit_bias: Optional[dict] = None):
# Endpoint URL for OpenAI API
openai_api_base = "http://0.0.0.0:65530/api/oai"
api_url = f"{openai_api_base}/v1/completions"

headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {api_key}'
}

# Payload with and without logit_bias
payload = {
"prompt": prompt,
"max_tokens": 3,
"model": "rwkv",
"temperature": 0
}

if logit_bias:
payload["logit_bias"] = logit_bias

response = requests.post(api_url, headers=headers, json=payload)

if response.status_code == 200:
print(json.dumps(response.json(), indent=2))
else:
print(f"Error {response.status_code}: {response.text}")

# Usage
API_KEY = "your-api-key" # Replace with your actual OpenAI API key

# Test with logit_bias
test_logit_bias(
api_key=API_KEY,
prompt="A profression that involves going to space is an ",
max_tokens=100,
logit_bias={15311: -100} # Replace with actual token values
)

# Test without logit_bias
print("Without")
test_logit_bias(
api_key=API_KEY,
prompt="A profression that involves going to space is an ",
max_tokens=100
)
```

This is the response I get.

```
{
"object": "text_completion",
"model": "assets/models/rwkv-4.safetensors",
"choices": [
{
"text": "\ud83d\ude80",
"index": 0,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 3,
"total_tokens": 14
}
}
Without
{
"object": "text_completion",
"model": "assets/models/rwkv-4.safetensors",
"choices": [
{
"text": "\ud83d\ude80",
"index": 0,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 11,
"completion_tokens": 3,
"total_tokens": 14
}
}
```

The weird thing is, i'm able to use the chat page in the UI just fine.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.