dssg / dssg/triage

Add an option for not persisting model objects to disk?

Open
#916 0 comments 0 reactions 0 assignees View on GitHub
needs-discussion
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

In the same way that running with `save_predictions=False` can be useful to avoid filling up database space, it would be useful to add optional behavior to delete model objects (maybe matrices as well?) from disk after they're done being used to save space avoid disk full issues during large, exploratory runs.

The main downside here, of course, is making it more expensive to resume an experiment after a crash mid-run, but it might be a reasonable trade-off.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by locating the persistence path for model objects and any existing save_predictions option, then determine the intended scope for models and matrices. Done should include a documented opt-out that removes the selected objects after use while preserving the stated resume trade-off.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.