CodeForPhilly / CodeForPhilly/balancer-main

[IMPROVE] Embedding model encoding time for search and file upload

Open
#441 0 comments 0 reactions 1 assignee Claimed by @sahilds1 View on GitHub
Dominant language
TypeScript
Stars
21
Forks
19
PR merge metrics
No merged PRs in 30d

Description

## Title: [IMPROVE] Embedding model encoding time for search and file upload

## Background

Search and file uploading call TransformerModel.get_instance() directly at request time

## Current State

https://github.com/CodeForPhilly/balancer-main/blob/75c1a14cb6759849c4987100236730b3355feffa/server/api/services/sentencetTransformer_model.py#L1-L22

## Acceptance Criteria
- [x] The model is warm before the first search or upload request hits

## Approach

Preload SentenceTransformer model at Django startup before traffic is routed to the application instance
Add tests for the embeddings services by pulling apart the core logic to make testing easier

## References

## Risks and Rollback

Fall back to lazy load using try except block if preloading embedding model at startup fails

## Screenshots / Recordings

## Related PR

Balancer PR https://github.com/CodeForPhilly/balancer-main/pull/461

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.