google-deepmind / google-deepmind/concordia

Resilient LLM API Wrapper with Exponential Backoff and Markdown Stripping

Open
#255 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.7k
Forks
367
PR merge metrics
No merged PRs in 30d

Description

**Problem**

Currently, Concordia's language model wrappers (in concordia/language_model/) pass raw strings directly from LLM providers to the simulation engine. This creates two primary points of failure that can crash an entire simulation mid-run:
Rate Limiting (HTTP 429): Large-scale simulations with multiple agents frequently hit API quotas. Currently, this results in an unhandled exception, causing loss of simulation state and wasted API costs for the preceding turns.
Formatting Noise: Modern LLMs often wrap responses in markdown code blocks.

Concrete Pain Point
A researcher running a 50-turn experiment with 10 agents might hit a rate limit on turn 45. Without a retry mechanism, the simulation fails, and the researcher must restart from turn 1, doubling the time and cost of the experiment.

Proposed Solution
I propose enhancing the base LanguageModel class or creating a decorator utility to provide a "Robust Call" layer:
``
1)Exponential Backoff: Use a retry strategy (similar to the tenacity pattern) to handle transient 429 or 500 errors.

2)Output Sanitization: A post-processing step to automatically strip triple backticks and leading/trailing whitespace before the text reaches the environment parser.

Proposed API Draft:
```
# In concordia/language_model/language_model.py

def sample_text_resilient(self, prompt: str, max_retries: int = 3) -> str:
"""Wrapper that handles retries and sanitizes markdown formatting."""
# 1. Implementation of exponential backoff logic
# 2. Call to self.sample_text()
# 3. Regex to strip ```markdown blocks```
# 4. Return clean string
```

**Why This Complements Existing Work**

Robustness: It directly addresses the root cause of issues #229 and #231 by sanitizing input before it hits the engine.
Cost Efficiency: Reduces the failure rate of long-running simulations, making the library more viable for "production-grade" research.
**My Plan**
I would like to work on this and can start by:

-Identifying the core sample_text calls in the Gemini and GPT wrappers.

-Implementing a lightweight retry utility (avoiding heavy new dependencies if possible).

-Adding unit tests that simulate "messy" LLM responses and API timeouts.

Contributor guide

Open the contributing guide

Research direction

Start with concordia/language_model/language_model.py, then locate the core sample_text calls in the Gemini and GPT wrappers. Review how existing wrappers surface API failures before choosing the retry utility. Done means transient 429 or 500 failures are retried with exponential backoff, markdown-wrapped responses are sanitized, and unit tests cover messy responses and API timeouts.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend-api-design
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.