Add Expected Calibration Error metrics and derived
Open
Nobody has claimed this yet.
enhancement
help wanted
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 726
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 5
Description
🚀 Feature
Idea is to provide Expected Calibration Error metrics as coded in Baal : https://github.com/ElementAI/baal/blob/master/src/baal/utils/metrics.py#L88
cc @Dref360
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with Baal's src/baal/utils/metrics.py around line 88, then inspect Ignite's existing metric entry points and related tests. Done means the requested Expected Calibration Error metrics and derived metric are available in Ignite and validated by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100