Support for survival tasks
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 755
- Forks
- 128
- PR merge metrics
- No merged PRs in 30d
Description
Description
Support for creating / uploading survival tasks is either (partially?) broken or at least insufficiently documented.
Steps/Code to Reproduce
An example survival task I would like to create has this format in xml:
<?xml version="1.0" encoding="UTF-8"?>
<oml:task_inputs xmlns:oml="http://openml.org/openml">
<oml:task_type_id>7</oml:task_type_id>
<oml:input name="source_data">46131</oml:input>
<oml:input name="target_feature_event">status</oml:input>
<oml:input name="target_feature_left"/>
<oml:input name="target_feature_right">time</oml:input>
<oml:input name="estimation_procedure">30</oml:input>
</oml:task_inputs>
based on dataset 46131 (gbsg) with target variable time (right-censored survival time) and status (binary censoring indicator, "event" variable).
I have tried various versions of this xml format, always based on what e.g. mlr3oml generates to publish tasks (which I successfully testes for classification tasks) and what I could find from existing survival tasks.
I attempted to create this task in two ways:
- Using R's
httrpackage and code analogous to whatmlr3omluses to upload tasks:
response = httr::POST(
url = "https://www.openml.org/api/v1/task",
body = list(
description = httr::upload_file("task.xml")
),
query = list(api_key = Sys.getenv("OPENMLAPIKEY"))
)
- A cURL command which I assume to be equivalent to the code above, but have limited experience with regarding file uploads:
curl -X POST "https://www.openml.org/api/v1/task?api_key=$OPENMLAPIKEY" \
-H "Accept: application/xml" \
-F "description=@task.xml;type=application/xml"
Expected Results
I expected to receive a task id of the correctly published survival task.
Actual Results
Versions of this error message:
<oml:error xmlns:oml="http://openml.org/openml">
<oml:code>619</oml:code>
<oml:message>Could not decode task inputs constraints json. Please contact developers.</oml:message>
<oml:additional_information>problematic input: target_feature_event</oml:additional_information>
</oml:error>
Additional remarks
- I have tried to use the same method to create a regression task based on this dataset through the API and I was successful, so I assume that the survival task is the issue at hand
- The
estimation_procedureI chose is30, but since these are specific to task types (why?) this is likely incorrect. I have tried a different estimation procedure that corresponds to survival tasks (19), but receive the same error response.
2.1 (I do, however, specifically need stratified 5-fold CV for my tasks) - @sebffischer tried to help me with this, and confirmed for me that technically creating survival tasks works on the test server but not the production server.
- As a side-note, when I tried to interactively use the REST API "try it out" feature on the website I also was not successful and from the server response below I assume that something more general was going wrong there, but I concede that I should probably open an issue on the website repo for that.
{
"error": {
"code": "611",
"message": "Description file not present"
}
}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the POST to /api/v1/task with the provided task.xml for task type 7, comparing production with the test server as noted in the issue. Inspect how survival-task input constraints handle target_feature_event and estimation_procedure. Done means a valid survival task is created and returns a task ID, including the requested stratified five-fold cross-validation setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, r
- Domain
- api, backend-api-design, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100