500 internal server error in web arena, leading to skewed results
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
Frequently, one of the results fails to generate. As seen in the screenshot, the left side failed to generate and there is no code, no error and no preview. It would be easy to simply upvote the right side model, but if this is due to a fault of LMArena (rate limiting for example), and not the model, then it will lead to skewed results.
This issue seems to be caused by a 500 internal server error.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file, test, or entry point is named. Start by reproducing the failed generation in the web arena and tracing the reported 500 response through the server logs and request path. Done means the failure is identified and addressed so one-sided failed results no longer skew evaluation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, web-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100