UMAP trained model consumes too much space
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 871
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 5
Description
Is there a way to reduce the space that a trained model takes? My trained model takes around 8 GB (~800k examples). I noticed that each model contains the embeddings of the train set, so its size is going to be proportional to the train set size. I'm not sure what else the model stores that explodes the size like that. Is it possible to not save those embeddings?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by investigating how a trained UMAP model stores the training-set embeddings and what other state contributes to the reported 8 GB size for roughly 800k examples. Determine whether those embeddings can be omitted while preserving the model's required behavior, and define completion as a substantially smaller saved model with its supported operations still working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100