openml / openml/OpenML

Problem with run/list filtered by run ids

Open
#723 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
755
Forks
128
PR merge metrics
No merged PRs in 30d

Description

    def test_get_runs_list(self):
        # TODO: comes from live, no such lists on test
        openml.config.server = self.production_server
        runs = openml.runs.list_runs(id=[2])
        self.assertEqual(len(runs), 1)
        for rid in runs:
            self._check_run(runs[rid])

For the following test, there is no run with id 2 in openml.org
https://www.openml.org/api/v1/run/list/run/2

However this shows that it exists in the live server:
https://www.openml.org/r/2
It happens even with the latest runs from the live server. Occurs also on the test server.
It also fails test_get_runs_list_by_filters.
test_get_runs_list_by_flow is also failing, although it has runs built on top of it.
I would say all the other run lists with different filters are failing because of this reason.

@janvanrijn @mfeurer

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with test_get_runs_list, test_get_runs_list_by_filters, and test_get_runs_list_by_flow, then compare their requests with the run/list/run/2 endpoint and the live run page. Done means the filtered run-list tests pass consistently on the live and test servers, including the expected run and flow results.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, python
Domain
api, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.