WithinSubjectEvaluation() and WithinDatasetEvaluation()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 264
- Avg merge
- 1d 13m
- Merged PRs (30d)
- 23
Description
I think these evaluation methods are much needed. You do not have them now, right?
WithinSubjectEvaluation() - evaluates the performance on all sessions for the same subject
WithinDatasetEvaluation() - it shuffles the data from all subjects (and sessions) and then it selects 1/5 for validation and the rest for training. Both training and validation will include data from all subjects. Results here will be more variable so it should be run several times as in cross validation.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the repository's existing evaluation methods and how datasets, subjects, sessions, training, and validation splits are represented. Determine where the two proposed evaluation modes belong and how repeated shuffled runs should be configured; done means both methods exist with the requested subject-wise and dataset-wide behavior and are covered by suitable tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100