Project-MONAI / Project-MONAI/MetricsReloaded
Evaluation suite - output
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 104
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
Context
Following the evaluation process on a set of references / output pairs, an aggregated report should be produced
Describe the solution you'd like
2 separate aggregations should be proposed:
- Raw aggregation - All the calculated metrics for each reference/output pair should be made available
- Population aggregation - Core statistics of the metrics over the population of reference/output pair should be produced including
- central tendency and variation
- indication of range
- indication of cases at 5, 25, 50, 75, 95 percentile of performance for each metric
- The reports should be made available as pandas data frame with the option to save in a specified format (csv, xlsx, html)
Additional context
https://github.com/Project-MONAI/MONAI/wiki/Evaluation-metrics-task-force
@jenspetersen
@danieltudosiu
@AReinke
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or implementation entry points are named. Start by reviewing the evaluation process and the linked Evaluation Metrics Task Force context, then identify where reference/output metrics are calculated; done means raw and population reports are available as pandas data frames and can be saved as CSV, XLSX, and HTML.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- analytics, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100