AOSSIE-Org / AOSSIE-Org/EduAid

[FEATURE]: Centralize device management and model loading across generator classes

オープン
#501 コメント 2 件 リアクション 0 件 担当者 0 名 GitHub で見る
enhancement
主要言語
JavaScript
スター
171
フォーク
423
PR マージ指標
30日以内にマージされた PR はありません

説明

### Feature and its Use Cases

## Summary

While exploring the backend architecture, I noticed that model loading and device detection logic are currently duplicated across multiple generator classes (MCQGenerator, ShortQGenerator, BoolQGenerator, ParaphraseGenerator, AnswerPredictor, QuestionGenerator, etc.).

Each class independently:

- Detects CUDA availability
- Creates its own `torch.device`
- Moves its model to that device
- In some cases, clears CUDA cache

Additionally, certain transformer checkpoints (e.g., T5-based models) appear to be loaded separately in multiple classes.

---

## Why This Could Be Improved

Although models are correctly instantiated once at application startup (which is good for performance), the current structure results in:

- Repeated device-detection logic across classes
- Increased risk of inconsistencies (e.g., missing `.to(device)` in certain scenarios)
- Multiple independent loads of similar transformer checkpoints
- Harder future scaling if GPU handling or model lifecycle changes

Centralizing this logic could improve:

- Maintainability
- Device consistency
- Memory efficiency (shared model instances where applicable)
- Future extensibility (multi-GPU support, lazy loading, batching, etc.)

---

## Proposed Direction (Open for Discussion)

A possible minimal approach:

1. Introduce a shared device utility (single source of truth for `torch.device`)
2. Optionally introduce a lightweight model registry/factory for shared checkpoints
3. Refactor generator classes to receive device/model references instead of redefining them

This would keep existing API behavior unchanged while improving internal structure.

---

## Use Cases

This change would make it easier to:

- Add new generator classes without duplicating hardware logic
- Modify GPU/CPU behavior from a single location
- Prevent device mismatch errors
- Optimize memory usage by sharing model instances
- Extend the system for future hardware scaling or deployment improvements

---

I would appreciate maintainer feedback on whether this direction aligns with the project's roadmap before proceeding with any refactoring work.

### Additional Context

_No response_

### Code of Conduct

- [x] I have joined the [Discord server](https://discord.gg/hjUhu33uAn) and will post updates there
- [x] I have searched existing issues to avoid duplicates

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。