gonczor / gonczor/DIYHelper

User uploaded document knowledge

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
1h 45m
Merged PRs (30d)
3

Description

Allow a user to upload a PDF and use it as explicit knowledge while answering questions. Uploaded documents form a persistent reusable library rather than temporary chat attachments.

## Example use case

1. The user downloads ATmega328P documentation.
2. The user uploads it to the document library.
3. The user selects it for a conversation.
4. The user asks how to blink an LED using the selected documentation.
5. The answer uses the PDF where applicable and retains the selection for follow-up questions.

## Agreed MVP design

- Use one persistent document model per unique PDF; do not create separate stored-file and user-upload models.
- Calculate a SHA-256 hash from the original PDF bytes while receiving the upload.
- Make the hash unique. Re-uploading identical bytes returns the existing document instead of storing or processing another copy.
- Store the original PDF using the existing storage abstraction.
- Extract and persist the complete document text, preserving page markers for page-level citations.
- Do not chunk or section-index documents in the MVP.
- Enforce a configurable maximum upload size.
- Let the user list and select previously uploaded documents.
- Associate selected document IDs with a conversation so the same documents remain available to follow-up questions.
- Send the complete extracted text of selected documents as answer context.
- If the combined selected documents exceed the prompt budget, ask the user to select fewer documents.
- Document deletion is out of scope for the MVP.

## Answer behavior

- Selected PDFs are explicit user-provided context and should be used where relevant.
- Answers should identify the selected document and page when supporting information can be traced to a page marker.
- The system must not claim that a fact or API appears in a PDF when it is supplied only by general model knowledge.
- General answer and follow-up behavior when references are missing is tracked separately in #6.

## Acceptance criteria

- In chat, a user can upload a PDF within the configured size limit.
- The original PDF is saved through an existing storage backend.
- Complete extracted text and page markers are persisted without chunking.
- Documents are saved and listed for reuse.
- The user can select previously uploaded documents for a conversation.
- Conversation document selection persists across follow-up questions.
- Answers receive the complete text of selected documents within the prompt budget.
- Answers can cite the selected document and relevant page.
- SHA-256 deduplication prevents storing and extracting identical PDF bytes multiple times.
- Uploading a duplicate returns the existing document.
- Oversized uploads are rejected with a clear validation error.
- Selecting documents whose combined content exceeds the prompt budget produces a clear request to select fewer documents.
- Upload, listing, selection, authentication, authorization, deduplication, size validation, persistence, and answer-context behavior are covered by tests.

## Out of scope

- Document deletion.
- Chunking, section retrieval, embeddings, and vector search.
- Multiple domain models representing the same uploaded document.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing storage abstraction and the chat answer flow, then trace how conversations persist their context. Implement the MVP across upload, extraction, document persistence, selection, authorization, deduplication, and answer context, using the acceptance criteria as the definition of done and adding coverage for each listed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.