EpistasisLab / EpistasisLab/tpot
scikit-learn-intelex integration
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
## Context
The Intel(R) Extension for Scikit-learn ([sklearnex](https://github.com/intel/scikit-learn-intelex/)) provides accelerations to popular classical machine learning algorithms, both on CPU and GPU. Given TPOT's heavy usage of scikit-learn algorithms, we believe there are compelling reasons for an integration of some sort with sklearnex's optimized regression and classification algorithms. Initial experimentation has shown potential for significant performance improvements - see this [jupyter notebook](https://colab.research.google.com/drive/1yeqbKg2Sv8affoHrOCsluunscjQnmqy5?usp=sharing) for further detail.


## Proposal
There are a few directions that this could go:
1. Integrate sklearnex into TPOT backend and allow users to set a `use_sklearnex` flag when initializing their TPOT classifier or regressor, in which case their config would use sklearnex implementations of algorithms instead of the default sklearn implementation (where possible). See an example of what this might look like in the code backend here: [fork](https://github.com/EpistasisLab/tpot/compare/master...ethanglaser:tpot:master) and how it could translate into performance improvements in the [notebook](https://colab.research.google.com/drive/1yeqbKg2Sv8affoHrOCsluunscjQnmqy5#scrollTo=h_TKDu-FnAFZ&line=5&uniqifier=1).
- Pros: any config could be accelerated here, no exclusion of algorithms (would use default sklearn implementation if an algorithm is not supported in sklearnex), relatively clean [integration ](https://github.com/EpistasisLab/tpot/compare/master...ethanglaser:tpot:master) as shown in the branch above
- Cons: configs and circumstances that do not lead to heavy usage of sklearnex-supported algorithms would not get significant performance improvements (i.e. sklearnex does not have an implementation for `neural_network.MLPClassifier`)
2. Create a separate sklearnex config classifier and regressor, which would yield a pipeline with sklearnex-supported algorithms (possibly something like [this regressor_config_dict_sklearnex](https://colab.research.google.com/drive/1yeqbKg2Sv8affoHrOCsluunscjQnmqy5#scrollTo=-8g-HMlYV2ph&line=7&uniqifier=1))
- Pros: all or most algorithms included in this config would be accelerated by sklearnex, yielding optimal performance improvements
- Cons: not all algorithms that a user might be interested in comparing would be covered by this config, and similarly - use of the existing TPOT configs that users are familiar with would not be accelerated
3. A combination of 1 and 2. Integrate into TPOT backend with a flag for users that want to accelerate existing configs, as well as a separate config focused on the sklearnex-accelerated algorithms.
- Pros: provides users with the **most flexibility** - can use the new config (option 2), accelerate existing configs (option 1), or use the original configs without accelerations as usual - fully backwards compatible
- Cons: none other than it would be the most involved integration (but still fairly simple)
In either case, there would be corresponding docs/tests updates and an additional tutorial created for a smooth integration, as well as any other additions you feel would be necessary.
Thank you for your consideration and look forward to continuing this discussion.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.