`SequentialFeatureSelector` is not passing pandas df to estimator/pipeline

Describe the bug

SequentialFeatureSelector cannot be used with a pipeline that expects a pandas dataframe (e.g. a one containing a ColumnTransformer) as an input.

At the same time the same pipeline can be used in cross_val_score.

Steps/Code to Reproduce

from sklearn.feature_selection import SequentialFeatureSelector
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler
from sklearn.compose import make_column_selector
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

import pandas as pd
import numpy as np

df = pd.DataFrame(data=np.random.rand(10,4), columns=["1","2", "3", "4"])
y = np.random.randint(0,2, size=(10,))

ct = ColumnTransformer(
    transformers=[
        ("numerical", StandardScaler(), make_column_selector(pattern="1")),
    ],
    remainder="passthrough",
)

pipeline = Pipeline([("ct", ct), ("lr", LogisticRegression())])

# this works
print("cross_val_score", cross_val_score(pipeline, df, y, cv=2))

sfs = SequentialFeatureSelector(pipeline, cv=2)
sfs.fit(df, y=y)

Expected Results

No error is thrown and the features are selected.

Actual Results

....sklearn/model_selection/_validation.py:372: FitFailedWarning: 
2 fits failed out of a total of 2.
...
ValueError: make_column_selector can only be applied to pandas dataframes

Versions

System:
    python: 3.9.5 (default, Nov 23 2021, 15:27:38)  [GCC 9.3.0]
executable: /root/.cache/pypoetry/virtualenvs/science-yd9zGFjU-py3.9/bin/python
   machine: Linux-5.13.0-35-generic-x86_64-with-glibc2.31

Python dependencies:
          pip: 22.0.3
   setuptools: 60.9.3
      sklearn: 1.0.2
        numpy: 1.19.5
        scipy: 1.8.0
       Cython: None
       pandas: 1.4.1
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

Start by running the provided SequentialFeatureSelector reproduction with the pandas DataFrame, ColumnTransformer, and pipeline. Trace where SequentialFeatureSelector prepares data for estimator fits and compare it with the working cross_val_score path. Done means the example completes without the pandas-dataframe error and selects features successfully.