Default target for multi task datasets
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 755
- Forks
- 128
- PR merge metrics
- No merged PRs in 30d
Description
Looking at the XSD, it occurred to me that the following field has some problems:
xs:element name="default_target_attribute" minOccurs="0" type="oml:casual_string1024"/> <!-- The default target attribute, if it exists. Can also have multiple values (comma-separated). Of course, tasks can be defined that use another attribute as target -->
I would propose to change this to:
xs:element name="default_target_attribute" minOccurs="0" max_occurs="unbounded" type="oml:casual_string1024"/> <!-- The default target attribute, if it exists.-->
For the following reasons:
- Comma separated is a suboptimal solution. I wouldn't be surprised if a comma is a valid entry in an arff attribute, which makes it impossible to parse several arff headers
- This distributes the burden of parsing to the specific libraries, java, weka, python, mlr and the evaluation engine all have to deal with this
- This currently requires quite a hack in the evaluation engine
- it is inconsistent with other fields, for example: creator, contributor, ignore attribute, etc.
Main question is how to solve this.
a) which libraries do depend on the feature that we can mark and upload this as a comma separated string? (AFAIK java and python not, but maybe mlr does)
b) How to fix the current fields in the database?
c) evaluation engine has hacked around this behavior, I would like to remove these hacks and set everything in such a way that it consistent with the other fields
d) ...
@joaquinvanschoren @berndbischl @mfeurer @giuseppec
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with openml_OS/views/pages/api_new/v1/xsd/openml.data.upload.xsd and trace how default_target_attribute is consumed by the Java, Weka, Python, and mlr libraries and the evaluation engine. Review the existing database fields and evaluation-engine workaround. Done means the compatibility impact, database migration, and consistent handling of multiple target attributes are agreed and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- xml
- Domain
- backend-api-design, databases, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100