openml / openml/OpenML

API occasionally unstable with not usable exceptions or strange HTTP error codes

Open
#1,083 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
755
Forks
128
PR merge metrics
No merged PRs in 30d

Description

We contact OpenML REST API inside our CI with few requests. Every so and so our tests fail. It seems that because the server is under the load. See for example this run: https://gitlab.com/datadrivendiscovery/d3m/-/jobs/924199064

So when using OpenML Python package we get:

openml.exceptions.OpenMLServerException: Database connection error. Usually due to high server load. Please wait for N seconds and try again. - None

Why it says N seconds? This is hard to wrap, wait, and retry? Or am I missing some information?

When using REST API directly, the error is:

requests.exceptions.HTTPError: 412 Client Error: Precondition Failed for url: https://www.openml.org/api/v1/json/task/168861

I do not think 412 is the right one here. 412 is used for cache validation. I think you want 429 Too Many Requests with Retry-After header. Or 503 Service Unavailable.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the REST API handling for /api/v1/json/task/168861 and the OpenML Python package's OpenMLServerException behavior. Reproduce the failure using the linked CI run and compare the 412 response with the reported 429 or 503 alternatives and Retry-After information. Done means transient server-load failures expose a usable retry signal and appropriate HTTP response.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, python
Domain
api, backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.