AOSSIE-Org / AOSSIE-Org/EduAid
[ENHANCEMENT]: Add Output Validation for LLM-generated Question Responses
- Ngôn ngữ chính
- JavaScript
- Star
- 171
- Fork
- 423
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
### Feature and its Use Cases
### Problem
LLM-based endpoints (`/get_mcq_llm`, `/get_boolq_llm`, `/get_shortq_llm`, `/get_problems_llm`) return generated questions without validating their structure or content.
As a result, the API may return HTTP 200 responses containing:
- Missing required fields (e.g., `options`, `answer`)
- Empty or malformed values
- Incomplete question sets
- Structurally inconsistent data
These issues are not detected before sending the response, leading to **silent failures** and unreliable outputs.
---
### Why This Matters
- Frontend may break due to undefined fields
- Users receive incomplete or unusable quizzes
- Debugging becomes difficult due to lack of validation
- Reduces overall reliability of LLM-based features
---
### Proposed Solution
Introduce a **lightweight output validation layer** for LLM-generated responses.
This layer should validate each generated question before it is returned by the API.
---
### Validation Rules
**MCQ**
- `question`: non-empty string
- `options`: non-empty list
- `correct_answer`: non-empty
**Short Questions**
- `question`: non-empty string
- `answer`: non-empty string
**Boolean Questions**
- `question`: non-empty string
- `answer`: must be `True` or `False`
---
### Expected Behavior
- Invalid questions are filtered out
- API responses contain only valid data
- No malformed or incomplete structures are returned
- Existing API behavior remains unchanged
---
### Scope
**In Scope**
- Validation of LLM-generated outputs
- Integration after parsing and before API response
**Out of Scope**
- Retry logic
- Pipeline/architecture changes
- Frontend changes
- Advanced schema validation systems
---
### Suggested Implementation
- Add a new module: `backend/Generator/output_validator.py`
- Implement simple validation functions per question type
- Integrate validation inside `llm_generator.py` before returning responses
---
### Impact
- Improves reliability of LLM endpoints
- Prevents silent failures
- Ensures consistent API contracts
- Enhances user experience
---
### Additional Context
### Additional Context
Input/request validation exists in parts of the system, but **LLM output validation is currently missing**, creating a gap in response correctness.
### Code of Conduct
- [x] I have joined the [Discord server](https://discord.gg/hjUhu33uAn) and will post updates there
- [x] I have searched existing issues to avoid duplicates
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.