agentscope-ai / agentscope-ai/OpenJudge

[Feature]: Add max_retries and timeout support to OpenaiChatModel

Abierto
#83 2 comentarios 0 reacciones 1 asignado Reclamado por @Wangzy455 Ver en GitHub
enhancement good first issue
Lenguaje dominante
Python
Estrellas
830
Forks
69
Merge medio
5 d 8 h
PR fusionados (30 d)
4

Descripción

## Description
Currently, the OpenaiChatModel class in OpenJudge does not explicitly handle network timeouts or transient API errors. If the OpenAI API hangs or returns a temporary 5xx error, the OpenaiChatModel might wait indefinitely or crash immediately.

## Suggested Implementation
The official OpenAI Python SDK (v1.x) natively supports max_retries and timeout parameters, but OpenJudge is not currently exposing or utilizing them in the model wrapper. This limits the robustness of the evaluation pipeline in unstable network environments.

I would like to expose the native max_retries and timeout parameters in the OpenaiChatModel initialization and pass them to the underlying OpenAI client.

### Implementation Steps:

#### Modify OpenaiChatModel:

Update the __init__ method in openjudge/models/openai_chat_model.py (or the relevant file path) to accept max_retries (int, default e.g., 2) and timeout (float, default e.g., 60.0).

Pass these parameters when initializing the AsyncOpenAI or OpenAI client within the class.

#### Add Unit Tests:

Create a test case in tests/models/test_openai_model.py.

Use unittest.mock to mock the OpenAI client.

Verify that when OpenaiChatModel is initialized with specific retry/timeout values, those values are correctly passed to the OpenAI client constructor.

Note: The test should not make real API calls; it should only verify parameter passing.

### Alternatives

We could implement a custom retry decorator (using libraries like tenacity) around the API call method. However, since the OpenAI SDK v1.x provides built-in support for retries and timeouts, using the native implementation is cleaner and requires fewer dependencies.

### Example usage after implementation:

```python
model = OpenaiChatModel(
model="gpt-4",
api_key="...",
timeout=30.0, # Fail if request takes longer than 30s
max_retries=3 # Retry 3 times on connection errors
)
```

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.