Standard deviation or other measures of error for AUC
- Dominant language
- Python
- Stars
- 304
- Forks
- 142
- PR merge metrics
- No merged PRs in 30d
Description
Is there a way to get some kind of error measurement (eg: standard deviation, confidence interval) of the AUC after doing cross validations? It would be helpful to get a list of all such values from doing cross validations instead of just a single AUC.
The relevant functions are AUC(test_results=None, multiclass=0, ignore_weights=False) and cross_validation(learners, examples, folds=10, stratified=StratifiedIfPossible, preprocessors=(), random_generator=0, callback=None, store_classifiers=False, store_examples=False).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the AUC(test_results=None, multiclass=0, ignore_weights=False) and cross_validation(learners, examples, folds=10, ...) entry points named in the issue. Trace how cross-validation results are collected and determine where per-fold AUC values and error measures would be exposed. Done means cross-validation can return the requested values alongside the single AUC, with behavior clarified for multiclass and weighted cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100