mlpack / mlpack/benchmarks

Expanding the benchmark coverage of this repository for all the toolkits

Open
#143 3 comments 4 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
107
Forks
46
PR merge metrics
No merged PRs in 30d

Description

@zoq @rcurtin I felt that this is a fantastic project where people can find which ml-toolkits are better for certain algorithms, and where the toolkits can improve themselves. So, I have been doing some work on my own that might be useful for this project. I have made a google sheet of the data I have been collecting in this regard. This google sheet contains:

  • the names of various machine learning and statistical analysis algorithms supported by the toolkits benchmarked in this repository
  • in which libraries they are found and in which libraries they aren't found
  • what are the API classes or functions that correspond to the algorithms
  • which algorithms have benchmarks and for what libraries are these benchmarks written.

I have till now covered all the algorithms provided by scikit-learn, mlpack and I am in the process of adding all the algorithms provided by Shogun into this list. This is a work in progress. I am going to add more algorithms to this list in the coming future and hopefully complete this. This is the google sheet that I am preparing:

image

image

image

image

I this regards I have some questions:
a) Is the aim of this project limited to benchmarking the algorithms supported mlpack? If no, I feel that having a sheet like this one, would help. (I got the idea of consolidating all this in a google sheet after I saw a google sheet on tensorflow's github when they were making tensorflow 2.0 and had to list all the API classes that needed some specific change).
b) Also, would it be possible for contributors from mlpack to also contribute to this sheet? I can give edit access. Currently, there are around 166 algorithms that are already listed with many more algorithms not covered and I haven't yet gone through all the library APIs. Would appreciate the help :)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked Google Sheet and the repository's existing benchmark coverage for scikit-learn, mlpack, and Shogun. No repository files, tests, or entry points are identified; first clarify whether the sheet should become a project artifact and agree on the required scope and completion criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.