jackwener / jackwener/OpenCLI

[Bug]: douyin search hangs/crashes under CDP automation even though the same profile passes manually

Open
#2,308 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
29.3k
Forks
2.9k
Avg merge
15h 36m
Merged PRs (30d)
70

Description

### Description

`opencli douyin search` fails consistently for a given query, with three
*different* error shapes across repeated runs on the same setup — but
manually navigating the exact same URL in the exact same Browser-Bridge-bound
Chrome profile works fine (full search results render, no verification wall).

This looks like Douyin's risk control treating CDP-driven navigation
differently from human-driven navigation on the same authenticated session,
rather than an account/login problem.

### Environment

- opencli v1.8.6 (doctor output below)
- Node v24.14.0
- macOS
- Extension v1.0.22, single connected profile

### Steps to Reproduce

```bash
opencli douyin search "工资为什么总不够花" --limit 5
```

### Observed failures (same command, different runs)

**Run 1 — MutationObserver crash:**
```
ok: false
error:
code: COMMAND_EXEC
message: >-
Douyin search extraction failed: TypeError: Failed to execute 'observe' on 'MutationObserver': parameter 1 is not of
type 'Node'.
at :43:14
at new Promise ()
at :1:1
exitCode: 1
```
This traces to `clis/douyin/search.js`, `WAIT_AND_EXTRACT_JS`:
```js
observer.observe(document.body, { childList: true, subtree: true });
```
`document.body` was `null` at the moment this ran — the only `.observe()`
call in the injected script, so the crash site is unambiguous.

**Run 2 — timeout:**
```
ok: false
error:
code: COMMAND_EXEC
message: Douyin search did not render result cards within the timeout. Open the same search in Chrome and verify login/security state before retrying.
exitCode: 1
```

**Manual check — no problem at all:**
Opened `https://www.douyin.com/search/工资为什么总不够花?type=video`
directly in the *same* Browser-Bridge-bound Chrome profile (confirmed via
the extension icon in that window, and `opencli profile list` shows a
single connected profile). Page rendered normally with 20+ real result
cards, no captcha, no login wall.

### `opencli doctor` output (ruling out Bridge/daemon connectivity)

```
opencli v1.8.6 doctor (node v24.14.0)

[OK] Daemon: running on port 19825 (v1.8.6)
[OK] Extension: connected (v1.0.22)

Profiles:
• h5r3c8v4: connected v1.0.22
[OK] Connectivity: connected in 1.3s

Everything looks good!
```

### Hypothesis

Daemon/extension/CDP connectivity is fine, and the same profile is not
blocked for a human-driven tab — so this doesn't look like a login/cookie
issue. It looks like Douyin's search endpoint serves a different (possibly
verification/interstitial) page state specifically when the tab is
CDP-controlled, and `WAIT_AND_EXTRACT_JS`'s state detector (`rendered` /
`login_wall` / `empty` / `timeout`) doesn't have a branch for that state —
so it either times out, or in one case hit a page structure where
`document.body` wasn't attached yet when `MutationObserver.observe()` ran.

### Suggested fixes

1. Defensive: guard `observer.observe(document.body, ...)` against a
`null`/not-yet-attached body (e.g. wait for `DOMContentLoaded` /poll for
`document.body` before constructing the observer) so this degrades to
`timeout` instead of an unhandled `TypeError`.
2. Investigate whether Douyin's search page reliably differentiates
CDP-driven vs. user-driven navigation on this domain, and if so, whether
there's a viable mitigation (e.g. matching the CDP session's execution
context flags more closely, or documenting this as a known limitation
for `douyin search` specifically).

Contributor guide

Open the contributing guide

Research direction

Start with clis/douyin/search.js and the WAIT_AND_EXTRACT_JS injected script, then reproduce with `opencli douyin search "工资为什么总不够花" --limit 5`. Check the MutationObserver setup and the rendered/login_wall/empty/timeout state handling against the reported runs. Done means the command no longer raises a null-body TypeError and reports a defined result or timeout state.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.