agentscope-ai / agentscope-ai/OpenJudge

[Feature]: Add max_retries and timeout support to OpenaiChatModel

Aperta
#83 2 commenti 0 reazioni 1 assegnatario Rivendicata da @Wangzy455 Vedi su GitHub
enhancement good first issue
Lingua principale
Python
Stelle
830
Fork
69
Merge medio
5g 8h
PR unite (30g)
4

Descrizione

## Description
Currently, the OpenaiChatModel class in OpenJudge does not explicitly handle network timeouts or transient API errors. If the OpenAI API hangs or returns a temporary 5xx error, the OpenaiChatModel might wait indefinitely or crash immediately.

## Suggested Implementation
The official OpenAI Python SDK (v1.x) natively supports max_retries and timeout parameters, but OpenJudge is not currently exposing or utilizing them in the model wrapper. This limits the robustness of the evaluation pipeline in unstable network environments.

I would like to expose the native max_retries and timeout parameters in the OpenaiChatModel initialization and pass them to the underlying OpenAI client.

### Implementation Steps:

#### Modify OpenaiChatModel:

Update the __init__ method in openjudge/models/openai_chat_model.py (or the relevant file path) to accept max_retries (int, default e.g., 2) and timeout (float, default e.g., 60.0).

Pass these parameters when initializing the AsyncOpenAI or OpenAI client within the class.

#### Add Unit Tests:

Create a test case in tests/models/test_openai_model.py.

Use unittest.mock to mock the OpenAI client.

Verify that when OpenaiChatModel is initialized with specific retry/timeout values, those values are correctly passed to the OpenAI client constructor.

Note: The test should not make real API calls; it should only verify parameter passing.

### Alternatives

We could implement a custom retry decorator (using libraries like tenacity) around the API call method. However, since the OpenAI SDK v1.x provides built-in support for retries and timeouts, using the native implementation is cleaner and requires fewer dependencies.

### Example usage after implementation:

```python
model = OpenaiChatModel(
model="gpt-4",
api_key="...",
timeout=30.0, # Fail if request takes longer than 30s
max_retries=3 # Retry 3 times on connection errors
)
```

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.