CodeForPhilly / CodeForPhilly/balancer-main

[IMPROVE] Embedding model encoding time for search and file upload

Aperta
#441 0 commenti 0 reazioni 1 assegnatario Rivendicata da @sahilds1 Vedi su GitHub
Lingua principale
TypeScript
Stelle
21
Fork
19
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Title: [IMPROVE] Embedding model encoding time for search and file upload

## Background

Search and file uploading call TransformerModel.get_instance() directly at request time

## Current State

https://github.com/CodeForPhilly/balancer-main/blob/75c1a14cb6759849c4987100236730b3355feffa/server/api/services/sentencetTransformer_model.py#L1-L22

## Acceptance Criteria
- [x] The model is warm before the first search or upload request hits

## Approach

Preload SentenceTransformer model at Django startup before traffic is routed to the application instance
Add tests for the embeddings services by pulling apart the core logic to make testing easier

## References

## Risks and Rollback

Fall back to lazy load using try except block if preloading embedding model at startup fails

## Screenshots / Recordings

## Related PR

Balancer PR https://github.com/CodeForPhilly/balancer-main/pull/461

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.