Add an option for not persisting model objects to disk?
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
In the same way that running with `save_predictions=False` can be useful to avoid filling up database space, it would be useful to add optional behavior to delete model objects (maybe matrices as well?) from disk after they're done being used to save space avoid disk full issues during large, exploratory runs.
The main downside here, of course, is making it more expensive to resume an experiment after a crash mid-run, but it might be a reasonable trade-off.
Contributor guide
Research direction
No files, tests, or entry points are named. Start by locating the persistence path for model objects and any existing save_predictions option, then determine the intended scope for models and matrices. Done should include a documented opt-out that removes the selected objects after use while preserving the stated resume trade-off.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100