AOSSIE-Org / AOSSIE-Org/EduAid

[FEATURE]: Add OCR Support for Image Uploads (.png, .jpg)

Aperta
#507 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
enhancement
Lingua principale
JavaScript
Stelle
171
Fork
425
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

### The Problem Context
Currently, the FileProcessor in EduAid successfully extracts text from .txt, .pdf, and .docx files. However, users frequently have study materials in the form of images (e.g., photos of textbook pages, screenshots, or whiteboard notes). Expanding the allowed file types to include images would make the question generation feature much more accessible.

### Proposed Direction (Open for Discussion)
A possible minimal approach:

- Integrate a robust OpenCV pipeline to preprocess uploaded images (converting to grayscale and applying binary thresholding to maximize text contrast).
- Use pytesseract (Tesseract OCR wrapper) to extract text from the processed images.
- Route the extracted text string directly into the existing NLP models, exactly as the system currently does for PDFs.

This would keep existing API behavior unchanged while expanding the system's input capabilities.

### Alternatives Considered
We could use cloud-based APIs (like Google Cloud Vision or AWS Textract), but keeping it local with OpenCV and PyTesseract avoids API costs and keeps the project fully open-source and self-hosted.

### Use Cases
This change would make it easier to:

- Support students who rely on photos of physical textbooks or handwritten notes.
- Quickly generate study questions from lecture screenshots or digital whiteboards.
- Broaden the accessibility of the platform without requiring users to manually transcribe their physical study materials.
- Avoid cloud API costs (like AWS Textract) by keeping the OCR pipeline completely local and open-source.

### Additional context
I understand adding Tesseract introduces a system-level dependency. I would appreciate maintainer feedback on whether this direction aligns with the project's roadmap before proceeding with any implementation work. If approved, I will ensure thorough testing is included and promptly address any CodeRabbit reviews.

### Code of Conduct

- [x] I have joined the [Discord server](https://discord.gg/hjUhu33uAn) and will post updates there
- [x] I have searched existing issues to avoid duplicates

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.