jackwener / jackwener/OpenCLI

Add `stream` command for chat-app adapters that surfaces the full SSE response (grok first)

Open
#1,847 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
29.5k
Forks
2.9k
Avg merge
15h 36m
Merged PRs (30d)
70

Description

## Problem

The existing ` ask` commands for chat-app adapters (`grok`, `chatgpt`, `claude`, `gemini`, `deepseek`) all wait for the assistant's next visible message bubble and return that text:

- `clis/grok/ask.js` polls `[data-testid="assistant-message"]` via `getMessageBubbles`
- `clis/chatgpt/ask.js`, `clis/claude/ask.js`, `clis/deepseek/ask.js`, `clis/gemini/ask.js` — same DOM-scrape shape

That works, but the assistant's underlying SSE response carries a lot the DOM doesn't:

- **thinking trace** (reasoning models hide it from the rendered bubble until the user clicks a toggle)
- **server-assigned `conversationId`/`responseId`/`parentResponseId`** (needed to chain follow-ups deterministically)
- **`model` / `modelHash`** (which exact variant served the answer — useful for grok-3 vs grok-4 routing)
- **`generatedImageUrls`** (the bubble only shows `` tags; the SSE has stable URLs the agent can save)
- **`title`** (Grok / DeepSeek / ChatGPT generate a title in the SSE tail)

For agents driving these UIs, missing this metadata means an extra round trip into the DOM or a fragile `read` call.

## Proposal

Add a ` stream` sub-command per chat adapter that:

1. Installs a `window.fetch` interceptor for the site's streaming chat endpoint
- Grok: `POST /rest/app-chat/conversations/new`
- ChatGPT: `POST /backend-api/f/conversation`
- Claude: `POST /api/organizations//chat_conversations//completion`
- Gemini: `POST /_/BardChatUi/.../StreamGenerate`
- DeepSeek: `POST /api/v0/chat/completion` (uses XHR, not fetch — also patches `XMLHttpRequest`)
2. Drains the response via `response.clone().body.getReader()` so SSE chunks are captured even while the site's own client SDK is consuming the body
3. Reuses the existing `sendMessage` / `ensureOn` / `isLoggedIn` helpers from `utils.js` to drive the UI
4. Parses the SSE/JSON-lines/wrb.fr/JSON-patch frames per site and returns ONE row with `response`, `thinking`, `model`, `conversationId`, `responseId`, `title`, `images`

Existing `ask` stays untouched — it's the right primitive for "just give me the rendered text"; `stream` is for callers that want the full envelope.

We aren't using `Strategy.INTERCEPT` / `page.installInterceptor()` because the upstream interceptor calls `await response.clone().json()`, which drops Grok's newline-delimited JSON, DeepSeek's `text/event-stream`, ChatGPT's JSON-patch SSE, etc. The fetch hook for `stream` keeps the raw body string and parses per site.

## Prior art

We've already built and tested this pattern against all five sites in a separate CLI: https://github.com/Daily-AC/webai-cli — single binary that wraps opencli's browser bridge. Each adapter's selectors, endpoint patterns, and SSE shape are documented there.

## Plan

- **This PR (#TBD)**: `clis/grok/stream.js` + `clis/grok/stream.test.js` only. Smallest reviewable surface; validates the pattern in the existing adapter style.
- **Follow-ups (one PR per site)**: chatgpt, claude, gemini, deepseek. Each parser is non-trivial enough to warrant its own PR.
- **Possible refactor later**: if all 5 land cleanly, extract a `_shared/stream-capture.js` helper for the fetch/XHR hook + drain pattern. Skipping that now to keep this PR minimal.

Happy to adjust naming (`stream` vs `ask-stream` vs `ask --api`), the columns shape, or anything else before expanding to the other four sites.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read clis/grok/ask.js and the existing utils.js helpers first, then review the Grok endpoint and the linked webai-cli prior art. Add only clis/grok/stream.js and clis/grok/stream.test.js, capturing the raw stream and returning one row with the listed fields while leaving ask unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli, web-dev
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.