AOSSIE-Org / AOSSIE-Org/EduAid

[Enhancement]: Implement Global Lazy Loading for Transformer Models to Reduce Memory Usage

Abierto
#525 1 comentario 0 reacciones 0 asignados Ver en GitHub
enhancement
Lenguaje dominante
JavaScript
Estrellas
171
Forks
425
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

### Feature and its Use Cases

Currently, multiple generator classes in the backend (MCQGenerator, ShortQGenerator, BoolQGenerator, AnswerPredictor, etc.) independently load transformer models using `from_pretrained()`.

This can lead to redundant model initialization and higher memory usage because the same models may be loaded multiple times across different components.

Proposed Feature:
Introduce a centralized lazy-loading model cache (for example, a ModelRegistry or model_cache module) that ensures each pretrained transformer model is loaded only once and reused across the application.

How it would work:
- Create a shared model registry or cache.
- Load a model only when it is first requested (lazy loading).
- Reuse the same instance across generator classes.

Benefits for users and developers:
- Reduced memory usage.
- Faster backend initialization.
- Improved scalability when handling concurrent requests.
- Cleaner backend architecture.

Use Cases:
1. When the backend initializes multiple generator classes that rely on the same transformer models.
2. When multiple API endpoints trigger model loading simultaneously.
3. When deploying EduAid on servers with limited memory resources.

### Additional Context

This enhancement would improve the backend performance and maintainability of EduAid by preventing redundant model loading and encouraging a more centralized architecture for managing transformer models.

### Code of Conduct

- [x] I have joined the [Discord server](https://discord.gg/hjUhu33uAn) and will post updates there
- [x] I have searched existing issues to avoid duplicates

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.