AOSSIE-Org / AOSSIE-Org/EduAid
[ENHANCEMENT]: Add Output Validation for LLM-generated Question Responses
- Lenguaje dominante
- JavaScript
- Estrellas
- 171
- Forks
- 423
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
### Feature and its Use Cases
### Problem
LLM-based endpoints (`/get_mcq_llm`, `/get_boolq_llm`, `/get_shortq_llm`, `/get_problems_llm`) return generated questions without validating their structure or content.
As a result, the API may return HTTP 200 responses containing:
- Missing required fields (e.g., `options`, `answer`)
- Empty or malformed values
- Incomplete question sets
- Structurally inconsistent data
These issues are not detected before sending the response, leading to **silent failures** and unreliable outputs.
---
### Why This Matters
- Frontend may break due to undefined fields
- Users receive incomplete or unusable quizzes
- Debugging becomes difficult due to lack of validation
- Reduces overall reliability of LLM-based features
---
### Proposed Solution
Introduce a **lightweight output validation layer** for LLM-generated responses.
This layer should validate each generated question before it is returned by the API.
---
### Validation Rules
**MCQ**
- `question`: non-empty string
- `options`: non-empty list
- `correct_answer`: non-empty
**Short Questions**
- `question`: non-empty string
- `answer`: non-empty string
**Boolean Questions**
- `question`: non-empty string
- `answer`: must be `True` or `False`
---
### Expected Behavior
- Invalid questions are filtered out
- API responses contain only valid data
- No malformed or incomplete structures are returned
- Existing API behavior remains unchanged
---
### Scope
**In Scope**
- Validation of LLM-generated outputs
- Integration after parsing and before API response
**Out of Scope**
- Retry logic
- Pipeline/architecture changes
- Frontend changes
- Advanced schema validation systems
---
### Suggested Implementation
- Add a new module: `backend/Generator/output_validator.py`
- Implement simple validation functions per question type
- Integrate validation inside `llm_generator.py` before returning responses
---
### Impact
- Improves reliability of LLM endpoints
- Prevents silent failures
- Ensures consistent API contracts
- Enhances user experience
---
### Additional Context
### Additional Context
Input/request validation exists in parts of the system, but **LLM output validation is currently missing**, creating a gap in response correctness.
### Code of Conduct
- [x] I have joined the [Discord server](https://discord.gg/hjUhu33uAn) and will post updates there
- [x] I have searched existing issues to avoid duplicates
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.