lablup / lablup/backend.ai

Accuracy badge for public ML models

Open
#13 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

Idea by @serialx.

- Let's provide "accuracy" badges for public ML models – visiting their repositories shows the accuracy value as a bdage in README (like CI build status badges)
- Feature ideas\* per-dataset / per-organization ranking
- per-commit history (i.e., how the accuracy has changed over time?)
- public dashboard to compare various open-source ML repositories

┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1159751085623729/1159757308089992) by [Unito](https://www.unito.io/learn-more)

JIRA Issue: BA-337

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. First clarify the scope among accuracy badges, dataset or organization rankings, per-commit history, and the comparison dashboard, then identify the relevant repository entry points. Done criteria should specify which of these features is being delivered and how model accuracy is measured.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.