ivadomed / ivadomed/model-spinal-rootlets

[Results] Cropped C2–T1 rootlets evaluation

Open
#111 15 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
8
Forks
2
PR merge metrics
No merged PRs in 30d

Description

## Summary

- Complete five-fold cross-validation and `fold_all` evaluation for the cropped C2–T1 model.
- Use the clean `Rootlets-train-test-labels` handoff.
- Compare `fold_all` with the released uncropped model on the same 17 held-out images.

## Checks

| Check | Result |
| --- | --- |
| Image/label pairs | 93/93 |
| Grid handling | 79 exact; 14 lossless axis/header fixes; no resampling |
| Crop input | image only |
| Crop padding | S/I 50/110, L/R 25/25, A/P 40/32 mm |
| Foreground retained | 93/93 |
| Orientation | RPI |
| CV coverage | 76/76 validated once |
| Runs | folds 0–4 + `fold_all` complete |
| Curves | native + per-class for all six runs |
| Complete-cohort outputs | 93/93 non-empty |

## Results

| Evaluation | Macro level Dice | Binary Dice |
| --- | ---: | ---: |
| Five-fold out-of-fold, n=76 | 0.618 ± 0.087 | 0.637 ± 0.083 |
| Released model, held-out n=17 | 0.635 ± 0.061 | 0.644 ± 0.059 |
| Cropped `fold_all`, held-out n=17 | 0.614 ± 0.064 | 0.625 ± 0.062 |
| Paired cropped − released | −0.021 ± 0.016 | −0.019 ± 0.015 |

| Arm | Prepare / crop | nnU-Net | Restore | End to end | Speedup |
| --- | ---: | ---: | ---: | ---: | ---: |
| Released | 33.40 s | 1101.75 s | 12.08 s | 1147.23 s | 1.00× |
| Cropped | 480.81 s | 513.66 s | 43.79 s | 1038.25 s | 1.10× |

## Method

- Patch `256 × 128 × 64`, spacing `0.70652 × 0.70000 × 0.70652 mm`, and batch size 2 were selected by nnU-Net.
- Dice is computed separately in 3-D for C2–T1, then averaged without level weights.
- A missed or false-positive level scores 0; a level absent from both masks is excluded.
- T1 means reference label value `9`; 92/93 references contain it.
- Legacy `_desc-rater*` and `_desc-staple` labels are excluded because they describe the older C2–C8 dorsal target.
- Plateau rule: minimum 1000 epochs; patience 400; minimum EMA gain 0.002; maximum 2000.

## Review

- [Full methods and tables](https://github.com/ivadomed/model-spinal-rootlets/blob/ky/107-cropped-rootlets-results/training/cervical_cropped/Dataset403_RESULTS.md)
- [Training curves](https://github.com/ivadomed/model-spinal-rootlets/tree/ky/107-cropped-rootlets-results/training/cervical_cropped/results/dataset403/training_curves)
- [Qualitative panels](https://github.com/ivadomed/model-spinal-rootlets/tree/ky/107-cropped-rootlets-results/training/cervical_cropped/results/dataset403/figures)
- [Machine-readable results](https://github.com/ivadomed/model-spinal-rootlets/tree/ky/107-cropped-rootlets-results/training/cervical_cropped/results/dataset403)
- Code: #108.
- Related: #105, #107, #109, [data-management#392](https://github.com/neuropoly/data-management/issues/392).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with training/cervical_cropped/Dataset403_RESULTS.md and the machine-readable outputs under training/cervical_cropped/results/dataset403. Review the methods, curves, figures, and held-out comparisons against the issue’s reported checks; done means the evaluation record is complete, reproducible, and consistent with code referenced in #108.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.