Running complex pipelines

Dear all,

I am a great fan of OpenML! It's great :)
Currently, I am looking into more complex custom generated Pipelines and unfortunately, I run into exceptions, e.g.:

  File "/home/felix/FastFeatures/new_project/fastsklearnfeature/feature_selection/ComplexityDrivenFeatureConstruction.py", line 497, in run
    my_run = openml.runs.run_model_on_task(my_pipeline, self.reader.task)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/runs/functions.py", line 44, in run_model_on_task
    flow = sklearn_to_flow(model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 43, in sklearn_to_flow
    rval = _serialize_model(o)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 393, in _serialize_model
    _extract_information_from_model(model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 501, in _extract_information_from_model
    rval = sklearn_to_flow(v, model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in sklearn_to_flow
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in <listcomp>
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in sklearn_to_flow
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in <listcomp>
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 43, in sklearn_to_flow
    rval = _serialize_model(o)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 393, in _serialize_model
    _extract_information_from_model(model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 501, in _extract_information_from_model
    rval = sklearn_to_flow(v, model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in sklearn_to_flow
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in <listcomp>
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in sklearn_to_flow
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in <listcomp>
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 43, in sklearn_to_flow
    rval = _serialize_model(o)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 393, in _serialize_model
    _extract_information_from_model(model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 501, in _extract_information_from_model
    rval = sklearn_to_flow(v, model)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in sklearn_to_flow
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in <listcomp>
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in sklearn_to_flow
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 46, in <listcomp>
    rval = [sklearn_to_flow(element, parent_model) for element in o]
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 43, in sklearn_to_flow
    rval = _serialize_model(o)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 397, in _serialize_model
    _check_multiple_occurence_of_component_in_flow(model, sub_components)
  File "/home/felix/FastFeatures/new_project/venv/lib/python3.7/site-packages/openml/flows/sklearn_converter.py", line 479, in _check_multiple_occurence_of_component_in_flow
    'trying to serialize %s.' % (visitee.name, model))
ValueError: Found a second occurence of component sklearn.compose._column_transformer.ColumnTransformer(identity=sklearn.preprocessing._function_transformer.FunctionTransformer) when trying to serialize FeatureUnion(n_jobs=None,
       transformer_list=[('deficit1557515560.9003851', Pipeline(memory=None,
     steps=[('deficit', ColumnTransformer(n_jobs=None, remainder='drop', sparse_threshold=0.3,
         transformer_weights=None,
         transformers=[('identity', FunctionTransformer(accept_sparse=False, check_inverse=True,
   ...        inverse_func=None, kw_args=None, pass_y='deprecated',
          validate=False), [0])]))]))],
       transformer_weights=None).

Is this a bug or a missing feature?

Best regards,
Felix

Start with openml/runs/functions.py and the sklearn_to_flow entry point in openml/flows/sklearn_converter.py. Trace the recursive calls through _serialize_model and _check_multiple_occurence_of_component_in_flow using the nested pipeline shown in the report. Done means complex custom pipelines can be serialized without the duplicate ColumnTransformer exception.