Migrate Embeddings Model to new Sandcastle Repo
- Dominant language
- JavaScript
- Stars
- 15.7k
- Forks
- 3.9k
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 34
Description
There has been some friction with using the semantic search embedding model directly from Huggingface. While we are in the process of migrating sandcastle into it's new repo; it would be a good time to begin inclusion of the embedding model within the new repo to help alleviate these issues for developers.
How this would look:
1. Copy all of the ONNX files from the huggingface page for the embedding modeL: [main](https://huggingface.co/avsolatorio/GIST-small-Embedding-v0/tree/main). Notably, when we include these files in Github, we may need to adjust configuration to support larger file sizes, as these embedding files can get quite large.
2. Update how we load the model in to target our repo source instead of huggingface. This will need to occur both at build time and at runtime (as we use the same model for both). Build time should be quite straightforward, but runtime we'll need to ensure that the load for these files doesn't block the loading of the application as a whole, as the larger file size may cause a notable increase in delay if the entire application waits for this file to be loaded.
Contributor guide
Research direction
Start with the Hugging Face GIST-small-Embedding-v0 model page and locate the current embedding-model loading paths used at build time and runtime. Move the ONNX files into the new Sandcastle repository, update both loading paths to use the repository source, and verify that runtime loading does not block application startup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, javascript
- Domain
- build-system, machine-learning, search
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100