Suggestion: Use gensim to load fastText word vectors
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 78
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
Hi Pieter,
It might make sense to move the code from the fastText module to gensim for loading the word vectors because it would make it possible to just load the word vectors (.vec file) instead of having to load the full model (.bin file). This will reduce both initialization time and memory usage. I have tried doing this, and it works quite well for my use case.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the fastText module and tracing how it currently loads .bin word-vector models. Review the proposed gensim-based .vec loading path and compare initialization time and memory use for the existing use case; done should mean the project can load .vec vectors without requiring the full model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100