EpistasisLab / EpistasisLab/tpot
Exporting pipelines to PMML/PFA
- Dominant language
- Jupyter Notebook
- Stars
- 10.1k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
I'm currently doing research in the area of model/pipeline persistence and came across [PMML](http://dmg.org/pmml/v4-2-1/GeneralStructure.html). It's basically an XML schema that lets you define all sorts of data mining/machine learning processes for both persistence and interoperability. It was specifically designed to decouple the tools that are used to _generate_ pipelines from the tools that are used to _apply_ them.
There are several libraries by [openscoring.io](https://github.com/jpmml) that can be used to export pipelines from popular environments such as sklearn, Apache Spark MLlib, R or XGBoost. Their counterparts allow for evaluation (i.e. execution) of said exported pipelines in e.g. plain Java, in a Spark context, an Android context or even in a database context such as PostgreSQL. The most interesting library for TPOT in particular should be [sklearn2pmml](https://github.com/jpmml/sklearn2pmml), which is a python wrapper around [jpmml-sklearn](https://github.com/jpmml/jpmml-sklearn) that converts pickled pipelines to PMML and is written in Java (talk about dependency hell).
I think that PMML (or its successor in the making [PFA](http://dmg.org/pfa/docs/motivation/)) would be a great format to use for persisting pipelines generated with TPOT as most people using TPOT will want to deploy the models "found" by it to some other platform. At least there seems to be some sort of demand for persisting models in general according to these issues #2, #11, #51, #65. Some of them suggest using python's pickle format, but I think a dedicated, platform independent solution should always be preferred. Not to talk about the [security issues that come with pickle](https://www.cs.uic.edu/~s/musings/pickle.html), that's a whole other story.
Excited to hear your thoughts on this!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.