uber / uber/causalml

cate_scoring default learner is oversized for its own per-fold, per-arm call pattern

Open
#1,037 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
6k
Forks
877
PR merge metrics
No merged PRs in 30d

Description

Low priority.

The default LGBMRegressor(num_leaves=64, ...) merged in #1030 (cate_scoring.py:63, :275, :399) is inherited from visualize.py's TMLE helpers (:343, :453, :696, :740), which fit on the full frame. dr_score / plug_in_t_score fit per-fold and per-arm — with the default n_folds=5 that is 10 fits on roughly n/10 rows each, where 64 leaves is well past what the slice supports. #1030 added verbose=-1, which silences the resulting "No further splits with positive gain" chatter without addressing the cause.

Two possible improvements, neither urgent:

  1. A smaller num_leaves for this call pattern, or one sized from the data.
  2. A docstring nudge on dr_score: comparing several candidate CATE models without an explicit learner refits 10 LGBM models per call, which is what compute_dr_pseudo_outcomes's own docstring advises avoiding by precomputing pseudo-outcomes once and passing pseudo_outcome_col.

Measured cost when #1030's three learner-less tests were added: tests/test_cate_scoring.py went from 17s to 37–55s.

Observed on master (477cd0a).

🤖 Generated with Claude Code

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with cate_scoring.py at lines 63, 275, and 399, then compare the learner usage with the TMLE helpers in visualize.py at lines 343, 453, 696, and 740. Read dr_score, plug_in_t_score, and compute_dr_pseudo_outcomes, and use tests/test_cate_scoring.py to measure the learner-less cases. Done means an agreed smaller or data-sized default, or a clear dr_score docstring warning, with the relevant tests still passing and the repeated-fit cost addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.