CodeForPhilly / CodeForPhilly/balancer-main
[IMPROVE] Embedding model encoding time for search and file upload
- Dominant language
- TypeScript
- Stars
- 21
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
## Title: [IMPROVE] Embedding model encoding time for search and file upload
## Background
Search and file uploading call TransformerModel.get_instance() directly at request time
## Current State
https://github.com/CodeForPhilly/balancer-main/blob/75c1a14cb6759849c4987100236730b3355feffa/server/api/services/sentencetTransformer_model.py#L1-L22
## Acceptance Criteria
- [x] The model is warm before the first search or upload request hits
## Approach
Preload SentenceTransformer model at Django startup before traffic is routed to the application instance
Add tests for the embeddings services by pulling apart the core logic to make testing easier
## References
## Risks and Rollback
Fall back to lazy load using try except block if preloading embedding model at startup fails
## Screenshots / Recordings
## Related PR
Balancer PR https://github.com/CodeForPhilly/balancer-main/pull/461
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.