coding-ninja-afk / coding-ninja-afk/ChatMyDocs

feat(api+ui): streaming responses (SSE) for /query

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
backend enhancement frontend good first issue
Dominant language
TypeScript
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

### Goal
Return assistant tokens as they are generated and render them live in the chat, ChatGPT-style.

### Scope
- Backend: add `POST /query/stream` that streams `text/event-stream` with events:
- `data: {"token":"...","done":false}` … `data: {"done":true,"sources":[...]}`
- Frontend: hook a `ReadableStream`/SSE client to incrementally append tokens.

### Notes (FastAPI + LangChain + Ollama)
- LangChain `ChatOllama` supports streaming via callbacks.
- Use `AsyncIteratorCallbackHandler` to yield tokens.
- Wrap with `StreamingResponse(media_type="text/event-stream").
- Keep current `/query` for non-streaming.

### Acceptance Criteria
- [ ] New endpoint `/query/stream` streams tokens.
- [ ] UI shows typing indicator and partial text updating.
- [ ] Cancelling the request (Esc or “Stop”) stops generation.
- [ ] Final `sources` appear identical to /query result.

### Pseudocode (backend)

```py
# app/main.py
from fastapi.responses import StreamingResponse
from langchain.callbacks import AsyncIteratorCallbackHandler
from langchain.schema.runnable import RunnableConfig

@app.post("/query/stream")
async def query_stream(req: QueryRequest):
if store.vectorstore._collection.count() == 0:
async def gen():
yield 'data: {"token":"No documents ingested yet.","done":true,"sources": []}\n\n'
return StreamingResponse(gen(), media_type="text/event-stream")

cb = AsyncIteratorCallbackHandler()
cfg: RunnableConfig = {"callbacks": [cb]}

async def agen():
# kick off generation in background
import anyio
async def run():
RAG_CHAIN.stream({"input": req.question}, cfg)
# stream tokens
async with anyio.create_task_group() as tg:
tg.start_soon(run)
async for t in cb.aiter():
yield f'data: {{"token":{t!r},"done":false}}\n\n'
yield f'data: {{"done":true,"sources":[]}}\n\n'

return StreamingResponse(agen(), media_type="text/event-stream")
````

### Pseudocode (frontend)

```ts
// api/rag.ts
export async function streamAnswer(q: string, onToken: (t:string)=>void) {
const res = await fetch(`${API}/query/stream`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ question: q }),
});
const reader = res.body!.getReader();
const dec = new TextDecoder();
let buf = "";
for (;;) {
const { value, done } = await reader.read();
if (done) break;
buf += dec.decode(value, { stream: true });
for (const block of buf.split("\n\n")) {
if (!block.startsWith("data:")) continue;
const payload = JSON.parse(block.slice(5).trim());
if (payload.token) onToken(payload.token);
if (payload.done) return payload.sources ?? [];
}
buf = "";
}
}
```

### Test Plan

* [ ] Long prompt: tokens arrive in multiple chunks.
* [ ] Cancel midway → no network errors, UI stops typing.
* [ ] Fallback: old `/query` still works.

```

---

## 2) Source snippet preview (side panel with highlighted chunk)

**Title**
```

feat(ui): source preview with snippet & page link

````

**Labels**: `enhancement`, `good first issue`, `frontend`

**Body**
```md
### Goal
Make each source clickable to show a short snippet and (if PDF) the page number.

### Scope
- Use already-returned `sources[]` with `{source, page}`.
- Show a right-side Drawer/Panel with:
- filename, page number
- chunk text (first ~400 chars of the chunk used)
- No PDF rendering yet (that’s a separate issue).

### Implementation idea
- In `ChatTranscript`, wrap each source in a button.
- On click, open a Drawer (Chakra UI) showing snippet.
- For snippet text: include a small `context` echo in response:
- On backend, attach `context[:400]` for each doc returned.

### Acceptance
- [ ] Each source opens a drawer with snippet and meta.
- [ ] Handles TXT/DOCX/PDF uniformly.
- [ ] Accessible focus trapping in Drawer.
````

---

## 3) Persist conversation history locally

**Title**

```
feat(ui): local conversation history (IndexedDB)
```

**Labels**: `enhancement`, `good first issue`, `frontend`

**Body**

````md
### Goal
Keep chats after refresh; allow switching between recent conversations.

### Scope
- IndexedDB (use `idb-keyval` or `localforage`)
- Schema:
```ts
type Chat = { id: string; createdAt: number; title: string; messages: Message[] };
````

* Save on every message; show a History list in Sidebar.

### Acceptance

* [ ] New chats auto-title by first user message (truncated).
* [ ] Click history item to load messages into current transcript.
* [ ] Delete chat works.
* [ ] No backend changes required.

### Notes

* Key by `chat:ID`.
* Keep max N (e.g., 20); drop oldest.

```

---

## 4) Multiple workspaces (collections)

**Title**
```

feat(workspaces): multiple collections with switcher

````

**Labels**: `enhancement`, `good first issue`, `backend`, `frontend`

**Body**
```md
### Goal
Support separate document sets (e.g., "Resume", "Research", "Manuals").

### Scope
- Backend: accept `workspace` string via header `X-Workspace` (or `?ws=` query).
- Use `Chroma(collection_name=f"cmd_{workspace}")`.
- Cache per-workspace Chroma instance (dict in `store.py`).
- Endpoints (`/ingest`, `/query`, `/stats`, `/reset`) read the workspace.

### Acceptance
- [ ] Workspace switcher UI (dropdown + “New…”).
- [ ] Each workspace has independent stats and chat.
- [ ] Reset affects only current workspace.

### Sketch (backend)
```py
# app/rag/store.py
_STORES: dict[str, Chroma] = {}

def get_store(ws: str) -> Chroma:
if ws not in _STORES:
_STORES[ws] = Chroma(collection_name=f"{settings.COLLECTION_NAME}_{ws}",
embedding_function=embeddings,
persist_directory=str(persist_dir))
return _STORES[ws]
````

### Tests

* [ ] Ingest in A does not change stats in B.
* [ ] Query in A returns only A’s sources.

```

---

## 5) PDF page thumbnails on hover (client-side with pdf.js)

**Title**
```

feat(ui): PDF page thumbnail preview on source hover

````

**Labels**: `enhancement`, `good first issue`, `frontend`

**Body**
```md
### Goal
When hovering a PDF source (with `page`), show a ~120px thumbnail of that page.

### Scope
- Use `pdfjs-dist` (client-side) to render a tiny canvas.
- Load the file via `` memory or via `/files/:name` if later added.
- Cache rendered thumbnails per `{filename,page}`.

### Acceptance
- [ ] Hover = popover with thumbnail + filename + page.
- [ ] Works in dark/light themes.
- [ ] Graceful fallback if file no longer available.

### Notes
- For first iteration, allow preview only for documents selected in this session (not persisted).
````

---

## 6) Keyboard UX: ↑ to edit last prompt, Enter to send, Shift+Enter for newline

**Title**

```
feat(ui): chat composer power keys (↑ edit, Enter send)
```

**Labels**: `enhancement`, `good first issue`, `frontend`, `UX`

**Body**

```md
### Goal
Match chat UX standards.

### Behavior
- Enter: send
- Shift+Enter: newline
- ArrowUp at empty composer: load last user message into composer for quick edit

### Acceptance
- [ ] Works in light/dark.
- [ ] No conflicts with IME (composition events respected).
- [ ] Has tooltip describing shortcuts.
```

---

## 7) Ingestion progress & error surfacing

**Title**

```
feat(ui): ingestion progress bar + error details
```

**Labels**: `enhancement`, `good first issue`, `frontend`, `backend`

**Body**

```md
### Goal
Make ingest status explicit (bytes saved → chunks split → vectors added).

### Scope
- Backend: send simple milestones in response JSON:
`{ "saved_bytes": N, "chunks": M, "status": "splitting" | "embedding" | "done" }`
(single response OK for first version)
- Frontend: stepper/progress bar in the Confirm panel.

### Acceptance
- [ ] Shows % based on milestones (33/66/100).
- [ ] Errors bubble up in a Toast with reason.
- [ ] Works for multi-file selection.
```

---

## 8) Voice input (browser speech-to-text)

**Title**

```
feat(ui): voice input for chat (Web Speech API)
```

**Labels**: `enhancement`, `good first issue`, `frontend`

**Body**

```md
### Goal
Press mic icon → dictate a question → it fills the composer.

### Scope
- Web Speech API (if available), graceful fallback otherwise.
- Mic button toggles listening; waveform anim optional.

### Acceptance
- [ ] Start/stop works, text lands in composer.
- [ ] Shows “listening…” status.
- [ ] No backend changes.
```

---

### Branch & commit guidance for contributors

* **Branch name**: `feat/streaming-sse`, `feat/source-snippet`, `feat/workspaces`, …
* **Commit style**:

* `feat(api): add /query/stream SSE endpoint`
* `feat(ui): incremental token rendering with SSE`
* `feat(ui): keyboard shortcuts (enter/shift+enter/up)`
* `fix(api): guard no-docs case for stream)`

Contributor guide

Open the contributing guide

Research direction

Start with the `/query` entry point in `app/main.py` and the `streamAnswer` function in `api/rag.ts`; trace the existing non-streaming flow before designing the SSE path. Done means `/query/stream` delivers token and final-source events, the UI renders and cancels partial responses, and the existing `/query` behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
fastapi, ollama, python, react, typescript
Domain
api, backend, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.