openml / openml/OpenML

Implement ROC, PR, Cost curves

Open
#733 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
755
Forks
128
PR merge metrics
No merged PRs in 30d

Description

Vishal volunteered to implement ROC curves on the run pages. I'm opening this issue to keep track of progress, and invite others to join.

The ViperCharts lib is ideal for doing this: http://viper.ijs.si/about/
We had an earlier implementation but this is not supported anymore:
https://github.com/openml/OpenML/blob/master/openml_OS/views/pages/frontend/r/subpage/run.php#L194

One important difference is that previously, we used the ViperCharts API (running in Ljubljana) to generate the plots and then pull them in. This time around, I would run ViperCharts on our server, as this is one dependency less to worry about.

Note: This should ideally be done in a JS-only way: get the data via ElasticSearch and push them to the ViperCharts library, so plots are created on the fly. I want to avoid PHP dependencies since the frontend will very likely be rewritten in Django+React. Just tell me what data you need for generating the plots, and I will add them to ElasticSearch.

Here is an example for doing JS-only plots for datasets:
https://github.com/openml/OpenML/blob/master/openml_OS/views/pages/frontend/d/javascript.php

The current JS for the run page contains a lot of legacy code, which I'll clean up asap:
https://github.com/openml/OpenML/blob/master/openml_OS/views/pages/frontend/r/javascript.php

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with openml_OS/views/pages/frontend/r/javascript.php and the dataset example in openml_OS/views/pages/frontend/d/javascript.php, then review the earlier run-page implementation. Determine the Elasticsearch data needed for ViperCharts and how to run the library on the server. Done means ROC, PR, and cost curves render on run pages through JavaScript without a PHP dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
data-visualization, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.