cate_scoring default learner is oversized for its own per-fold, per-arm call pattern
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6k
- Forks
- 877
- PR merge metrics
- No merged PRs in 30d
Description
Low priority.
The default LGBMRegressor(num_leaves=64, ...) merged in #1030 (cate_scoring.py:63, :275, :399) is inherited from visualize.py's TMLE helpers (:343, :453, :696, :740), which fit on the full frame. dr_score / plug_in_t_score fit per-fold and per-arm — with the default n_folds=5 that is 10 fits on roughly n/10 rows each, where 64 leaves is well past what the slice supports. #1030 added verbose=-1, which silences the resulting "No further splits with positive gain" chatter without addressing the cause.
Two possible improvements, neither urgent:
- A smaller
num_leavesfor this call pattern, or one sized from the data. - A docstring nudge on
dr_score: comparing several candidate CATE models without an explicit learner refits 10 LGBM models per call, which is whatcompute_dr_pseudo_outcomes's own docstring advises avoiding by precomputing pseudo-outcomes once and passingpseudo_outcome_col.
Measured cost when #1030's three learner-less tests were added: tests/test_cate_scoring.py went from 17s to 37–55s.
Observed on master (477cd0a).
🤖 Generated with Claude Code
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with cate_scoring.py at lines 63, 275, and 399, then compare the learner usage with the TMLE helpers in visualize.py at lines 343, 453, 696, and 740. Read dr_score, plug_in_t_score, and compute_dr_pseudo_outcomes, and use tests/test_cate_scoring.py to measure the learner-less cases. Done means an agreed smaller or data-sized default, or a clear dr_score docstring warning, with the relevant tests still passing and the repeated-fit cost addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100