[Bug]: Console pages fail to load on slow networks — all-in-one MB-level API responses (skills list & chat history, uncompressed) vs fixed 30 s frontend timeout

# [Bug]: Console pages fail to load on slow networks — all-in-one MB-level API responses (skills list & chat history, uncompressed) vs fixed 30 s frontend timeout

## QwenPaw Version

2.0.1 (pip install)

## Description

Several console views fail to load whenever the underlying API response cannot be transferred within the console's fixed 30 s fetch timeout. All affected endpoints share the same pattern: **a single response returns the entire dataset (no pagination / full bodies embedded), sent uncompressed**. Whether a view fails correlates exactly with payload size:

**Affected endpoint family 1 — skills lists** (Workspace → Skills page, Settings → Skill Pool page):

| Endpoint | Dataset | Response size (uncompressed) | Result on a ~25 KB/s link |
|---|---|---|---|
| `GET /api/skills` | 1 skill | ~5 KB | ✅ loads |
| `GET /api/skills` | 26 skills | ~212 KB | ✅ loads |
| `GET /api/skills` | 70 skills | ~880 KB | ❌ 30 s timeout → "加载技能失败" |
| `GET /api/skills` | 36 skills | ~1.36 MB | ❌ 30 s timeout |
| `GET /api/skills/workspaces` (needed by Skill Pool page) | all workspaces | ~5.3 MB | ❌ 30 s timeout even at ~60 KB/s |

**Affected endpoint family 2 — chat history** (Chat page, opening a conversation):

`GET /api/chats/{chat_id}` (`src/qwenpaw/app/chats/api.py:232`) returns **all messages of the chat in one response — no pagination / limit / incremental loading**. Measured on real workspaces:

| Agent | Chats | Median size | Largest | Result on a ~25 KB/s link (before fix) |
|---|---|---|---|---|
| default | 33 | ~241 KB | ~1.02 MB (679 messages) | large chats always time out |
| Equity-Alpha-Scout | 40 | ~404 KB | ~1.33 MB | large chats always time out |

Opening a ~1 MB chat history on a ~25 KB/s link needs ~40 s+ → the frontend aborts at 30 s, the conversation never opens. On faster-but-still-modest links it is just very slow.

Root causes, in combination:

1. **All-in-one responses.**
- `SkillSpec` extends `SkillInfo`, whose `content: str` field carries the *entire* SKILL.md body of every skill. `_build_workspace_skill_specs()` in `src/qwenpaw/app/routers/skills.py` dumps it into `GET /api/skills`; `list_workspace_skill_sources()` (`GET /api/skills/workspaces`) repeats this for *every* workspace, multiplying the payload. The `content` field alone accounts for 40–50 % of the response, while the list view only needs metadata.
- `GET /api/chats/{chat_id}` has no pagination parameters and returns the full message history; long-running chats grow monotonically (~1 MB+ each here).
2. **Responses are not compressed.** The FastAPI app (`src/qwenpaw/app/_app.py`) registers CORS/Auth/AgentContext middleware but no `GZipMiddleware`, so multi-MB JSON (highly compressible Markdown/text) is transferred as-is (`content-length: 1362873`, no `content-encoding`).
3. **The console fetch wrapper has a hard 30 s timeout with 0 retries** (default `timeout: 3e4, retries: 0` in the API client), after which it aborts and the page shows a load-failure error. The server access log still records `200 OK` (bytes are handed to the kernel quickly), so the failure is invisible from server-side logs — only the client ever sees it.

**Security considerations:** none.

## Component(s) Affected

- [x] Core / Backend (app, agents, config, providers, utils, local_models)
- [x] Console (frontend web UI)
- [x] Skills

## Environment

- **QwenPaw version:** 2.0.1
- **OS:** Linux (Ubuntu, kernel 6.8.0-101-generic, x86_64), cloud VPS accessed over a congested international link
- **Install method:** pip (venv)
- **Python version:** 3.12

## Steps to Reproduce

1. Have a workspace with many skills (or a few large ones) so that `GET /api/skills` exceeds ~1 MB (e.g. 36 skills → 1.36 MB here), and/or a long chat whose `GET /api/chats/{chat_id}` is ~1 MB.
2. Open the console over a slow/unstable network (or throttle with Chrome DevTools / CDP to ~25 KB/s, 250 ms latency).
3. Open **Workspace → Skills**, or **Settings → Skill Pool**, or open a large conversation in the **Chat** page.

## Actual vs Expected

- **Actual:** The list/history request stays pending and is aborted by the frontend at exactly 30 000 ms (`AbortError`, `Request timeout after 30000ms: GET /skills`); the page shows a load-failure toast and stays empty (for chats: the conversation cannot be opened). Server access log shows `200 OK` for the same requests, masking the failure.
- **Expected:** Lists and histories load regardless of network quality — list views should not require transferring full bodies/histories; large JSON responses should be compressed; the fetch layer should tolerate slow transfers (retry / longer timeout / pagination).

## Logs / Screenshots

Measured with an in-browser fetch replicating the console's own wrapper (30 s AbortController), network throttled to 25 KB/s:

```
[skills, workspace A ~5KB ] OK 200 items=1 in 1646ms
[skills, workspace B ~212KB ] OK 200 items=26 in 8922ms
[skills, workspace C ~880KB ] ABORTED/TIMEOUT after 30000ms: AbortError <-- page shows load failure
[skills, workspace D ~1.36MB] ABORTED/TIMEOUT after 30000ms: AbortError
[skills/workspaces ~5.3MB ] ABORTED/TIMEOUT after 30000ms: AbortError
```

Chat history sizes (uncompressed, `GET /api/chats/{id}`):

```
default: 33 chats, median 241KB, max 1.02MB
Equity-Alpha-Scout: 40 chats, median 404KB, max 1.33MB
```

Response headers showing no compression:

```
HTTP/1.1 200 OK
content-length: 1362873
(no content-encoding)
```

**Validation of the gzip direction (patched locally):** after adding `GZipMiddleware` to `src/qwenpaw/app/_app.py`, the same endpoints measure: skills 880 KB→321 KB / 1.36 MB→345 KB, workspaces 5.3 MB→1.6 MB, chat history 1.07 MB→232 KB (~2.7–4.6×). Re-running the throttled test, the two previously-failing skills pages now load in ~13 s (well under the timeout), and chats that previously never opened now load.

## Additional Notes

Suggested directions (any one helps; together they fix it robustly):

1. Slim the list responses / paginate the history:
- Make `GET /api/skills` and `GET /api/skills/workspaces` return metadata only (drop `content`, and possibly `scripts`/`references`); load the full body lazily when the skill drawer is opened (a dedicated `GET /skills/{name}/content`-style endpoint).
- Add pagination / incremental loading to `GET /api/chats/{chat_id}` (e.g. return the latest N messages, fetch older ones on scroll).
2. Add `GZipMiddleware` (or equivalent) for API responses — verified locally to compress these payloads ~2.7–4.6× and recover the failing pages (see above).
3. Increase the default timeout / add retries (or paginate) for the affected endpoints in the console API client.

Workarounds for affected users: reduce the number of installed skills per workspace; archive/delete old large chats; access the console over a faster link.

---

_Disclosure per the contribution policy: root-cause analysis and this draft were assisted by an AI agent (Qwenpaw+Kimi K3); the behavior was reproduced programmatically (throttled headless-browser test) on a real 2.0.1 deployment, and environment details above are genuine._