AOSSIE-Org / AOSSIE-Org/EduAid
[FEATURE]: Add OCR Support for Image Uploads (.png, .jpg)
- 主要語言
- JavaScript
- 星號
- 171
- 分支
- 425
- PR 合併指標
- 30 天內沒有已合併 PR
描述
### The Problem Context
Currently, the FileProcessor in EduAid successfully extracts text from .txt, .pdf, and .docx files. However, users frequently have study materials in the form of images (e.g., photos of textbook pages, screenshots, or whiteboard notes). Expanding the allowed file types to include images would make the question generation feature much more accessible.
### Proposed Direction (Open for Discussion)
A possible minimal approach:
- Integrate a robust OpenCV pipeline to preprocess uploaded images (converting to grayscale and applying binary thresholding to maximize text contrast).
- Use pytesseract (Tesseract OCR wrapper) to extract text from the processed images.
- Route the extracted text string directly into the existing NLP models, exactly as the system currently does for PDFs.
This would keep existing API behavior unchanged while expanding the system's input capabilities.
### Alternatives Considered
We could use cloud-based APIs (like Google Cloud Vision or AWS Textract), but keeping it local with OpenCV and PyTesseract avoids API costs and keeps the project fully open-source and self-hosted.
### Use Cases
This change would make it easier to:
- Support students who rely on photos of physical textbooks or handwritten notes.
- Quickly generate study questions from lecture screenshots or digital whiteboards.
- Broaden the accessibility of the platform without requiring users to manually transcribe their physical study materials.
- Avoid cloud API costs (like AWS Textract) by keeping the OCR pipeline completely local and open-source.
### Additional context
I understand adding Tesseract introduces a system-level dependency. I would appreciate maintainer feedback on whether this direction aligns with the project's roadmap before proceeding with any implementation work. If approved, I will ensure thorough testing is included and promptly address any CodeRabbit reviews.
### Code of Conduct
- [x] I have joined the [Discord server](https://discord.gg/hjUhu33uAn) and will post updates there
- [x] I have searched existing issues to avoid duplicates
貢獻指南
這個儲存庫沒有索引到貢獻指南
評估
這個 Issue 還沒有評估資料。