scikit-learn / scikit-learn/scikit-learn
Add random forest row-subsampling without replacement
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
Describe the workflow you want to enable
I do appreciate the current option of disabling bootstrapping via the Boolean argument bootstrap.
However, currently there is only one alternative: If False, the whole (identical) dataset is used to build each tree.
There is well known research though (starting with the paper by Strobl et al. from 2007) that in certain situations subsamples drawn without replacement leads to better performance.
Many well known random forest implementations (such as ranger ) offer this sub sampling as an alternative to the bootstrap.
I would greatly appreciate it if sklearn would offer the same.
Describe your proposed solution
For both RandomForestRegressor and RandomForestClassifier allow the user to draw subsamples without replacement for each tree instead of the bootstrap.
The user can choose the fraction of sub samples drawn for each tree (default: 0.632 )
Ideally the functions _generate_unsampled_indices and _generate_sampled_indices would still work.
Describe alternatives you've considered, if relevant
No response
Additional context
No response
API considerations
Currently, random forests have the option bootstap=True or False. If this new feature of tree-wise row-subsampling without replacement is added, there are several options:
- Add a new option
with_replacement=True(default), orFalse, that only takes effect ifbootstrap=True.
Disadvantage: The term bootstrap explicitly means sampling with replacement. - Add a new option
row_subsampling=True(default) orFalse, which samples with replacement ifbootstrap=Trueand without replacement ifbootstrap=False.
Disadvantage: It would change current behaviour forbootstrap=False, which currently means no sampling at all. - Add new option
sampling="bootstrap"(default), and allow callable / splitter to be passed. Deprecate optionbootstrap(proposed in https://github.com/scikit-learn/scikit-learn/issues/20953#issuecomment-923957749).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review RandomForestRegressor and RandomForestClassifier, along with the _generate_unsampled_indices and _generate_sampled_indices helpers named in the issue. Compare the proposed sampling API alternatives and establish tests for per-tree sampling without replacement, configurable sample fractions, and preserved existing bootstrap behavior; done means both estimators support the agreed API and behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100