ORF estimate takes too long
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.8k
- Forks
- 827
- PR merge metrics
- No merged PRs in 30d
Description
I am using the DiscreteTreatmentOrthoForest's const_marginal_effect() method to estimate my test data, which is 4840 by 24. I used a trained model and call model.const_marginal_effect(test_data) directly. It takes hours to estimated the effect. I am wondering if I am doing correctly or there is any way to speed up the process? Here is my code for reference. Thank you so much!
#Load trained model
model = pickle.load(open('ModelPath', 'rb'))
#Load test data, shape (4840, 24)
test_data = pickle.load(open('TestDataPath', 'rb')).values
#Estimate the treatment effect
TE = model.const_marginal_effect(test_data)
And here is the first few lines of terminal prints:
[Parallel(n_jobs=-1)]: Using backend ThreadingBackend with 4 concurrent workers.
[Parallel(n_jobs=-1)]: Done 24 tasks | elapsed: 9.9min
[Parallel(n_jobs=-1)]: Done 120 tasks | elapsed: 54.4min
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with DiscreteTreatmentOrthoForest.const_marginal_effect() and the supplied 4,840×24 test-data example, then review the terminal's Parallel timing output. Done should establish whether the call is being used correctly and identify a supported way to reduce the runtime, if one exists.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 22/100