mlcommons/medperf

View on GitHub

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.

Stars: 172
Forks: 36
Open beginner issues: 0
Indexed issues: 16

Dominant language: Python
License: Apache-2.0
Last GitHub push: Aug 4, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Beginner friendly Everything

Open All

16 open issues indexed

[BUG] Documented install `pip install -e ./cli` fails with resolution-too-deep on pip 25.1+ Open

Difficulty 4/5 3-5 days Newbie friendliness 48/100

mlcommons/medperf#704 · Opened Jul 30, 2026
Synthetic clinical safety companion benchmark proposal Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlcommons/medperf#699 · Opened Jul 8, 2026
Proposal: optional audit-safe metadata for federated evaluation runs Open

Difficulty 2/5 1-3 hours Newbie friendliness 68/100

mlcommons/medperf#688 · Opened May 11, 2026
Documentation updates Open

topic: documentation

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/medperf#635 · Opened Feb 12, 2025
What does EXTRA-MODALITIES mean? Open

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/medperf#592 · Opened May 14, 2024
[BUG] mutable object as default values Open

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/medperf#587 · Opened Apr 30, 2024
[BUG] client-side associations filtering Open

component: client project: Core type: bug

Difficulty 4/5 3-5 days Newbie friendliness 25/100

mlcommons/medperf#584 · Opened Apr 24, 2024
[BUG] `dataset submit` only validates fields after doing a bunch more stuff Open

component: client project: Core topic: UI/UX type: bug type: enhancement

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/medperf#581 · 1 comment · Opened Apr 17, 2024
[FEATURE] Download tables as CSV from Admin Dashboard Open

component: server topic: misc

Difficulty 4/5 3-5 days Newbie friendliness 32/100

mlcommons/medperf#548 · Opened Feb 26, 2024
[FEATURE] Ask for medperf storage location on installation Open

component: client topic: misc

Difficulty 3/5 1-2 days Newbie friendliness 35/100

mlcommons/medperf#547 · Opened Feb 23, 2024
Add input checks on config parameters Open

project: Core

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/medperf#525 · Opened Jan 30, 2024
Determine what endpoints should return 404 when permission is denied Open

component: server project: Core project: RANO topic: misc

Difficulty 4/5 3-5 days Newbie friendliness 35/100

mlcommons/medperf#515 · 1 comment · Opened Dec 1, 2023
Define how results can be edited Open

component: server project: Core topic: misc

Difficulty 5/5 Over a week Newbie friendliness 25/100

mlcommons/medperf#510 · Opened Nov 24, 2023
[FEATURE] Have a contributing guide Open

project: Core topic: documentation type: enhancement

Difficulty 3/5 1-2 days Newbie friendliness 45/100

mlcommons/medperf#498 · Opened Nov 6, 2023
[FEATURE] Improve readme.md Open

project: Core topic: documentation type: enhancement

mlcommons/medperf#487 · 1 reaction · 2 assignees · Opened Sep 13, 2023
[FEATURE] Update CLI functions documentation Open

component: client project: Core topic: documentation type: enhancement

mlcommons/medperf#296 · 2 comments · 1 assignee · Opened Oct 24, 2022

mlcommons/medperf

Get new issues in your inbox