[Bug]: douyin search hangs/crashes under CDP automation even though the same profile passes manually
- Dominant language
- JavaScript
- Stars
- 29.3k
- Forks
- 2.9k
- Avg merge
- 15h 36m
- Merged PRs (30d)
- 70
Description
### Description
`opencli douyin search` fails consistently for a given query, with three
*different* error shapes across repeated runs on the same setup — but
manually navigating the exact same URL in the exact same Browser-Bridge-bound
Chrome profile works fine (full search results render, no verification wall).
This looks like Douyin's risk control treating CDP-driven navigation
differently from human-driven navigation on the same authenticated session,
rather than an account/login problem.
### Environment
- opencli v1.8.6 (doctor output below)
- Node v24.14.0
- macOS
- Extension v1.0.22, single connected profile
### Steps to Reproduce
```bash
opencli douyin search "工资为什么总不够花" --limit 5
```
### Observed failures (same command, different runs)
**Run 1 — MutationObserver crash:**
```
ok: false
error:
code: COMMAND_EXEC
message: >-
Douyin search extraction failed: TypeError: Failed to execute 'observe' on 'MutationObserver': parameter 1 is not of
type 'Node'.
at :43:14
at new Promise ()
at :1:1
exitCode: 1
```
This traces to `clis/douyin/search.js`, `WAIT_AND_EXTRACT_JS`:
```js
observer.observe(document.body, { childList: true, subtree: true });
```
`document.body` was `null` at the moment this ran — the only `.observe()`
call in the injected script, so the crash site is unambiguous.
**Run 2 — timeout:**
```
ok: false
error:
code: COMMAND_EXEC
message: Douyin search did not render result cards within the timeout. Open the same search in Chrome and verify login/security state before retrying.
exitCode: 1
```
**Manual check — no problem at all:**
Opened `https://www.douyin.com/search/工资为什么总不够花?type=video`
directly in the *same* Browser-Bridge-bound Chrome profile (confirmed via
the extension icon in that window, and `opencli profile list` shows a
single connected profile). Page rendered normally with 20+ real result
cards, no captcha, no login wall.
### `opencli doctor` output (ruling out Bridge/daemon connectivity)
```
opencli v1.8.6 doctor (node v24.14.0)
[OK] Daemon: running on port 19825 (v1.8.6)
[OK] Extension: connected (v1.0.22)
Profiles:
• h5r3c8v4: connected v1.0.22
[OK] Connectivity: connected in 1.3s
Everything looks good!
```
### Hypothesis
Daemon/extension/CDP connectivity is fine, and the same profile is not
blocked for a human-driven tab — so this doesn't look like a login/cookie
issue. It looks like Douyin's search endpoint serves a different (possibly
verification/interstitial) page state specifically when the tab is
CDP-controlled, and `WAIT_AND_EXTRACT_JS`'s state detector (`rendered` /
`login_wall` / `empty` / `timeout`) doesn't have a branch for that state —
so it either times out, or in one case hit a page structure where
`document.body` wasn't attached yet when `MutationObserver.observe()` ran.
### Suggested fixes
1. Defensive: guard `observer.observe(document.body, ...)` against a
`null`/not-yet-attached body (e.g. wait for `DOMContentLoaded` /poll for
`document.body` before constructing the observer) so this degrades to
`timeout` instead of an unhandled `TypeError`.
2. Investigate whether Douyin's search page reliably differentiates
CDP-driven vs. user-driven navigation on this domain, and if so, whether
there's a viable mitigation (e.g. matching the CDP session's execution
context flags more closely, or documenting this as a known limitation
for `douyin search` specifically).
Contributor guide
Research direction
Start with clis/douyin/search.js and the WAIT_AND_EXTRACT_JS injected script, then reproduce with `opencli douyin search "工资为什么总不够花" --limit 5`. Check the MutationObserver setup and the rendered/login_wall/empty/timeout state handling against the reported runs. Done means the command no longer raises a null-body TypeError and reports a defined result or timeout state.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100