coding-ninja-afk / coding-ninja-afk/ChatMyDocs
feat(api+ui): streaming responses (SSE) for /query
- Dominant language
- TypeScript
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
### Goal
Return assistant tokens as they are generated and render them live in the chat, ChatGPT-style.
### Scope
- Backend: add `POST /query/stream` that streams `text/event-stream` with events:
- `data: {"token":"...","done":false}` … `data: {"done":true,"sources":[...]}`
- Frontend: hook a `ReadableStream`/SSE client to incrementally append tokens.
### Notes (FastAPI + LangChain + Ollama)
- LangChain `ChatOllama` supports streaming via callbacks.
- Use `AsyncIteratorCallbackHandler` to yield tokens.
- Wrap with `StreamingResponse(media_type="text/event-stream").
- Keep current `/query` for non-streaming.
### Acceptance Criteria
- [ ] New endpoint `/query/stream` streams tokens.
- [ ] UI shows typing indicator and partial text updating.
- [ ] Cancelling the request (Esc or “Stop”) stops generation.
- [ ] Final `sources` appear identical to /query result.
### Pseudocode (backend)
```py
# app/main.py
from fastapi.responses import StreamingResponse
from langchain.callbacks import AsyncIteratorCallbackHandler
from langchain.schema.runnable import RunnableConfig
@app.post("/query/stream")
async def query_stream(req: QueryRequest):
if store.vectorstore._collection.count() == 0:
async def gen():
yield 'data: {"token":"No documents ingested yet.","done":true,"sources": []}\n\n'
return StreamingResponse(gen(), media_type="text/event-stream")
cb = AsyncIteratorCallbackHandler()
cfg: RunnableConfig = {"callbacks": [cb]}
async def agen():
# kick off generation in background
import anyio
async def run():
RAG_CHAIN.stream({"input": req.question}, cfg)
# stream tokens
async with anyio.create_task_group() as tg:
tg.start_soon(run)
async for t in cb.aiter():
yield f'data: {{"token":{t!r},"done":false}}\n\n'
yield f'data: {{"done":true,"sources":[]}}\n\n'
return StreamingResponse(agen(), media_type="text/event-stream")
````
### Pseudocode (frontend)
```ts
// api/rag.ts
export async function streamAnswer(q: string, onToken: (t:string)=>void) {
const res = await fetch(`${API}/query/stream`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ question: q }),
});
const reader = res.body!.getReader();
const dec = new TextDecoder();
let buf = "";
for (;;) {
const { value, done } = await reader.read();
if (done) break;
buf += dec.decode(value, { stream: true });
for (const block of buf.split("\n\n")) {
if (!block.startsWith("data:")) continue;
const payload = JSON.parse(block.slice(5).trim());
if (payload.token) onToken(payload.token);
if (payload.done) return payload.sources ?? [];
}
buf = "";
}
}
```
### Test Plan
* [ ] Long prompt: tokens arrive in multiple chunks.
* [ ] Cancel midway → no network errors, UI stops typing.
* [ ] Fallback: old `/query` still works.
```
---
## 2) Source snippet preview (side panel with highlighted chunk)
**Title**
```
feat(ui): source preview with snippet & page link
````
**Labels**: `enhancement`, `good first issue`, `frontend`
**Body**
```md
### Goal
Make each source clickable to show a short snippet and (if PDF) the page number.
### Scope
- Use already-returned `sources[]` with `{source, page}`.
- Show a right-side Drawer/Panel with:
- filename, page number
- chunk text (first ~400 chars of the chunk used)
- No PDF rendering yet (that’s a separate issue).
### Implementation idea
- In `ChatTranscript`, wrap each source in a button.
- On click, open a Drawer (Chakra UI) showing snippet.
- For snippet text: include a small `context` echo in response:
- On backend, attach `context[:400]` for each doc returned.
### Acceptance
- [ ] Each source opens a drawer with snippet and meta.
- [ ] Handles TXT/DOCX/PDF uniformly.
- [ ] Accessible focus trapping in Drawer.
````
---
## 3) Persist conversation history locally
**Title**
```
feat(ui): local conversation history (IndexedDB)
```
**Labels**: `enhancement`, `good first issue`, `frontend`
**Body**
````md
### Goal
Keep chats after refresh; allow switching between recent conversations.
### Scope
- IndexedDB (use `idb-keyval` or `localforage`)
- Schema:
```ts
type Chat = { id: string; createdAt: number; title: string; messages: Message[] };
````
* Save on every message; show a History list in Sidebar.
### Acceptance
* [ ] New chats auto-title by first user message (truncated).
* [ ] Click history item to load messages into current transcript.
* [ ] Delete chat works.
* [ ] No backend changes required.
### Notes
* Key by `chat:ID`.
* Keep max N (e.g., 20); drop oldest.
```
---
## 4) Multiple workspaces (collections)
**Title**
```
feat(workspaces): multiple collections with switcher
````
**Labels**: `enhancement`, `good first issue`, `backend`, `frontend`
**Body**
```md
### Goal
Support separate document sets (e.g., "Resume", "Research", "Manuals").
### Scope
- Backend: accept `workspace` string via header `X-Workspace` (or `?ws=` query).
- Use `Chroma(collection_name=f"cmd_{workspace}")`.
- Cache per-workspace Chroma instance (dict in `store.py`).
- Endpoints (`/ingest`, `/query`, `/stats`, `/reset`) read the workspace.
### Acceptance
- [ ] Workspace switcher UI (dropdown + “New…”).
- [ ] Each workspace has independent stats and chat.
- [ ] Reset affects only current workspace.
### Sketch (backend)
```py
# app/rag/store.py
_STORES: dict[str, Chroma] = {}
def get_store(ws: str) -> Chroma:
if ws not in _STORES:
_STORES[ws] = Chroma(collection_name=f"{settings.COLLECTION_NAME}_{ws}",
embedding_function=embeddings,
persist_directory=str(persist_dir))
return _STORES[ws]
````
### Tests
* [ ] Ingest in A does not change stats in B.
* [ ] Query in A returns only A’s sources.
```
---
## 5) PDF page thumbnails on hover (client-side with pdf.js)
**Title**
```
feat(ui): PDF page thumbnail preview on source hover
````
**Labels**: `enhancement`, `good first issue`, `frontend`
**Body**
```md
### Goal
When hovering a PDF source (with `page`), show a ~120px thumbnail of that page.
### Scope
- Use `pdfjs-dist` (client-side) to render a tiny canvas.
- Load the file via `` memory or via `/files/:name` if later added.
- Cache rendered thumbnails per `{filename,page}`.
### Acceptance
- [ ] Hover = popover with thumbnail + filename + page.
- [ ] Works in dark/light themes.
- [ ] Graceful fallback if file no longer available.
### Notes
- For first iteration, allow preview only for documents selected in this session (not persisted).
````
---
## 6) Keyboard UX: ↑ to edit last prompt, Enter to send, Shift+Enter for newline
**Title**
```
feat(ui): chat composer power keys (↑ edit, Enter send)
```
**Labels**: `enhancement`, `good first issue`, `frontend`, `UX`
**Body**
```md
### Goal
Match chat UX standards.
### Behavior
- Enter: send
- Shift+Enter: newline
- ArrowUp at empty composer: load last user message into composer for quick edit
### Acceptance
- [ ] Works in light/dark.
- [ ] No conflicts with IME (composition events respected).
- [ ] Has tooltip describing shortcuts.
```
---
## 7) Ingestion progress & error surfacing
**Title**
```
feat(ui): ingestion progress bar + error details
```
**Labels**: `enhancement`, `good first issue`, `frontend`, `backend`
**Body**
```md
### Goal
Make ingest status explicit (bytes saved → chunks split → vectors added).
### Scope
- Backend: send simple milestones in response JSON:
`{ "saved_bytes": N, "chunks": M, "status": "splitting" | "embedding" | "done" }`
(single response OK for first version)
- Frontend: stepper/progress bar in the Confirm panel.
### Acceptance
- [ ] Shows % based on milestones (33/66/100).
- [ ] Errors bubble up in a Toast with reason.
- [ ] Works for multi-file selection.
```
---
## 8) Voice input (browser speech-to-text)
**Title**
```
feat(ui): voice input for chat (Web Speech API)
```
**Labels**: `enhancement`, `good first issue`, `frontend`
**Body**
```md
### Goal
Press mic icon → dictate a question → it fills the composer.
### Scope
- Web Speech API (if available), graceful fallback otherwise.
- Mic button toggles listening; waveform anim optional.
### Acceptance
- [ ] Start/stop works, text lands in composer.
- [ ] Shows “listening…” status.
- [ ] No backend changes.
```
---
### Branch & commit guidance for contributors
* **Branch name**: `feat/streaming-sse`, `feat/source-snippet`, `feat/workspaces`, …
* **Commit style**:
* `feat(api): add /query/stream SSE endpoint`
* `feat(ui): incremental token rendering with SSE`
* `feat(ui): keyboard shortcuts (enter/shift+enter/up)`
* `fix(api): guard no-docs case for stream)`
Contributor guide
Research direction
Start with the `/query` entry point in `app/main.py` and the `streamAnswer` function in `api/rag.ts`; trace the existing non-streaming flow before designing the SSE path. Done means `/query/stream` delivers token and final-source events, the UI renders and cancels partial responses, and the existing `/query` behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, ollama, python, react, typescript
- Domain
- api, backend, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100