EducationalTestingService / EducationalTestingService/skll

Consider parallelizing xval

Open
#523 0 comments 0 reactions 1 assignee Claimed by @aoifecahill View on GitHub
enhancement
Dominant language
Python
Stars
561
Forks
69
PR merge metrics
No merged PRs in 30d

Description

Cross-validation runs serially (grid search cross-validation, however, does make use of threads). This is a considerable bottleneck for large data-sets/large feature spaces. For example, in recent experiments with 15k samples and perhaps up to 100k features, 10-fold cross-validation can take upwards of **two weeks**. It would be a good idea to consider parallelizing at the cross-validation fold-level, if possible. For example, perhaps each fold can be `gridmap`ed individually or folds can be run in threads (however, as mentioned, grid search cross-validation already spawns 3 threads, so that would have to be kept in mind).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.