CodeForPhilly / CodeForPhilly/balancer-main
[IMPROVE] Embedding model encoding time for search and file upload
- Lingua principale
- TypeScript
- Stelle
- 21
- Fork
- 19
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
## Title: [IMPROVE] Embedding model encoding time for search and file upload
## Background
Search and file uploading call TransformerModel.get_instance() directly at request time
## Current State
https://github.com/CodeForPhilly/balancer-main/blob/75c1a14cb6759849c4987100236730b3355feffa/server/api/services/sentencetTransformer_model.py#L1-L22
## Acceptance Criteria
- [x] The model is warm before the first search or upload request hits
## Approach
Preload SentenceTransformer model at Django startup before traffic is routed to the application instance
Add tests for the embeddings services by pulling apart the core logic to make testing easier
## References
## Risks and Rollback
Fall back to lazy load using try except block if preloading embedding model at startup fails
## Screenshots / Recordings
## Related PR
Balancer PR https://github.com/CodeForPhilly/balancer-main/pull/461
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.