EpistasisLab / EpistasisLab/tpot

Exporting pipelines to PMML/PFA

Open
#152 44 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Jupyter Notebook
Stars
10.1k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

I'm currently doing research in the area of model/pipeline persistence and came across [PMML](http://dmg.org/pmml/v4-2-1/GeneralStructure.html). It's basically an XML schema that lets you define all sorts of data mining/machine learning processes for both persistence and interoperability. It was specifically designed to decouple the tools that are used to _generate_ pipelines from the tools that are used to _apply_ them.

There are several libraries by [openscoring.io](https://github.com/jpmml) that can be used to export pipelines from popular environments such as sklearn, Apache Spark MLlib, R or XGBoost. Their counterparts allow for evaluation (i.e. execution) of said exported pipelines in e.g. plain Java, in a Spark context, an Android context or even in a database context such as PostgreSQL. The most interesting library for TPOT in particular should be [sklearn2pmml](https://github.com/jpmml/sklearn2pmml), which is a python wrapper around [jpmml-sklearn](https://github.com/jpmml/jpmml-sklearn) that converts pickled pipelines to PMML and is written in Java (talk about dependency hell).

I think that PMML (or its successor in the making [PFA](http://dmg.org/pfa/docs/motivation/)) would be a great format to use for persisting pipelines generated with TPOT as most people using TPOT will want to deploy the models "found" by it to some other platform. At least there seems to be some sort of demand for persisting models in general according to these issues #2, #11, #51, #65. Some of them suggest using python's pickle format, but I think a dedicated, platform independent solution should always be preferred. Not to talk about the [security issues that come with pickle](https://www.cs.uic.edu/~s/musings/pickle.html), that's a whole other story.

Excited to hear your thoughts on this!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.