User uploaded document knowledge
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 45m
- Merged PRs (30d)
- 3
Description
Allow a user to upload a PDF and use it as explicit knowledge while answering questions. Uploaded documents form a persistent reusable library rather than temporary chat attachments.
## Example use case
1. The user downloads ATmega328P documentation.
2. The user uploads it to the document library.
3. The user selects it for a conversation.
4. The user asks how to blink an LED using the selected documentation.
5. The answer uses the PDF where applicable and retains the selection for follow-up questions.
## Agreed MVP design
- Use one persistent document model per unique PDF; do not create separate stored-file and user-upload models.
- Calculate a SHA-256 hash from the original PDF bytes while receiving the upload.
- Make the hash unique. Re-uploading identical bytes returns the existing document instead of storing or processing another copy.
- Store the original PDF using the existing storage abstraction.
- Extract and persist the complete document text, preserving page markers for page-level citations.
- Do not chunk or section-index documents in the MVP.
- Enforce a configurable maximum upload size.
- Let the user list and select previously uploaded documents.
- Associate selected document IDs with a conversation so the same documents remain available to follow-up questions.
- Send the complete extracted text of selected documents as answer context.
- If the combined selected documents exceed the prompt budget, ask the user to select fewer documents.
- Document deletion is out of scope for the MVP.
## Answer behavior
- Selected PDFs are explicit user-provided context and should be used where relevant.
- Answers should identify the selected document and page when supporting information can be traced to a page marker.
- The system must not claim that a fact or API appears in a PDF when it is supplied only by general model knowledge.
- General answer and follow-up behavior when references are missing is tracked separately in #6.
## Acceptance criteria
- In chat, a user can upload a PDF within the configured size limit.
- The original PDF is saved through an existing storage backend.
- Complete extracted text and page markers are persisted without chunking.
- Documents are saved and listed for reuse.
- The user can select previously uploaded documents for a conversation.
- Conversation document selection persists across follow-up questions.
- Answers receive the complete text of selected documents within the prompt budget.
- Answers can cite the selected document and relevant page.
- SHA-256 deduplication prevents storing and extracting identical PDF bytes multiple times.
- Uploading a duplicate returns the existing document.
- Oversized uploads are rejected with a clear validation error.
- Selecting documents whose combined content exceeds the prompt budget produces a clear request to select fewer documents.
- Upload, listing, selection, authentication, authorization, deduplication, size validation, persistence, and answer-context behavior are covered by tests.
## Out of scope
- Document deletion.
- Chunking, section retrieval, embeddings, and vector search.
- Multiple domain models representing the same uploaded document.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the existing storage abstraction and the chat answer flow, then trace how conversations persist their context. Implement the MVP across upload, extraction, document persistence, selection, authorization, deduplication, and answer context, using the acceptance criteria as the definition of done and adding coverage for each listed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100