openml / openml/OpenML

redundant estimation procedures in DB

Open
#1,216 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
755
Forks
128
PR merge metrics
No merged PRs in 30d

Description

In data/sql/estimation_procedure.sql:
(5, 1, '10% Holdout set', 'holdout', 1, NULL, 'false', 33, 'true', 'false', '2014-12-31 21:00:00'),
(6, 1, '33% Holdout set', 'holdout', 1, NULL, 'false', 33, 'true', 'false', '2014-12-31 21:00:00'),

Surely the first 33 ought to be 10?

These records (and possibly others) seem redundant (and the 10% case has also a wrong percentage field):
(27, 2, 'Test on Training Data', 'testontrainingdata', NULL, NULL, 'false', NULL, NULL, 'false', '2019-03-16 11:30:14'),
(29, 9, '10-fold Crossvalidation', 'crossvalidation', 1, 10, 'false', NULL, 'true', 'false', '2014-12-31 20:00:00'),
(30, 10, '10-fold Crossvalidation', 'crossvalidation', 1, 10, 'false', NULL, 'true', 'false', '2023-02-22 11:46:54'),
(31, 10, '5 times 2-fold Crossvalidation', 'crossvalidation', 5, 2, 'false', NULL, 'true', 'false', '2023-02-22 11:46:54'),
(32, 10, '10 times 10-fold Crossvalidation', 'crossvalidation', 10, 10, 'false', NULL, 'true', 'false', '2023-02-22 11:46:54'),
(33, 10, '10% Holdout set', 'holdout', 1, NULL, 'false', 33, 'true', 'false', '2023-02-22 11:46:54'),
(34, 10, '33% Holdout set', 'holdout', 1, NULL, 'false', 33, 'true', 'false', '2023-02-22 11:46:54'),
(35, 11, '33% Holdout set', 'holdout', 1, NULL, 'false', 33, 'true', 'false', '2023-06-15 16:34:54');

Also, dead link in https://github.com/openml/OpenML/blob/d2f1cab55d6bea95e9067c1a54af3a2b04b966ae/CONTRIBUTING.md

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing data/sql/estimation_procedure.sql and the dead link in CONTRIBUTING.md. Compare the listed estimation records to identify redundant entries and incorrect percentages, then update the affected records and repair the link; done means the data is non-redundant, the 10% values are correct, and the documentation link works.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases, documentation
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.