Colsample performance when using tree_method=hist

In general, I would expect usage of colsample_by* parameters to improve training speed (per tree), since we do not need to consider all features when evaluating splits. For my use case, however, I do not observe this.

Using tree_method=hist and grow_policy=depthwise, I see that most of the time is taken in building histograms (I profiled QuantileHistMaker::Builder::ExpandWithDepthWise, and saw that almost all of the time is spent in BuildLocalHistograms), which is actually done before sampling the feature sets for each node (in https://github.com/dmlc/xgboost/blob/522b8977c27b422a4cdbe1ecc59a4d57a5df2c36/src/tree/updater_quantile_hist.cc#L996).

Could performance be improved by sampling the features prior to building the histograms instead, since we then do not need to compute histograms for the unused features? If so, can we please include this as a feature request?