scikit-learn / scikit-learn/scikit-learn

Interaction terms between categorical and numerical features

Open
#15,263 5 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

module:compose module:preprocessing
Dominant language
Python
Stars
67.3k
Forks
27.4k
Avg merge
1d 15h
Merged PRs (30d)
58

Description

Description

While it is possible to create feature interactions with the same individual preprocessing in ColumnTransformer via PolynomialFeatures, I find no (convincing) solution for interactions of features with different individual preprocessing, e.g. categorical column with a continuous numerical column.
Such interactions might improve models from sklearn.linear_model.

Code Example
import pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, PolynomialFeatures


df = pd.DataFrame({'a': ['red', 'red', 'blue', 'blue'],
                   'b': ['high', 'low', 'high', 'low'],
                   'x': [1, 1, 1, 2],
                   'y': [2, 3, 4, 2]
                  })

# interactions for features with no individual preprocessing works fine,
# i.e. numerical ones
column_trans = ColumnTransformer(
    [('xy_num',
      PolynomialFeatures(degree=2, interaction_only=True, include_bias=False),
      ['x', 'y'])],
     remainder='drop')
column_trans.fit_transform(df)

# interactions for ohe encoded also works with helper function
cat_cat = make_pipeline(
    OneHotEncoder(),
    PolynomialFeatures(degree=2, interaction_only=True, include_bias=False)
)
column_trans = ColumnTransformer(
    [('ab_cat', cat_cat, ['a', 'b'])],
     remainder='drop')
column_trans.fit_transform(df)
Expected Results
# no clue for interactions between one-hot-encoded 'a' and 'x' 
column_trans = ColumnTransformer(
    [('a_x',
      magic_pipeline(OneHotEncoder(), 'passthrough'),
      ['a', 'x'])],
     remainder='drop')
Versions

sklearn version 0.21

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the reported ColumnTransformer and PolynomialFeatures examples, focusing on how categorical and numerical columns receive different preprocessing. Determine whether the requested interaction support can be specified from the existing examples, and define completion as a documented or implemented way to create interactions between one-hot-encoded categorical features and numerical features.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.