NanmiCoder / NanmiCoder/MediaCrawler

mac 抖音 爬取报错

Open
#576 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
65.3k
Forks
12.6k
PR merge metrics
No merged PRs in 30d

Description

python3 main.py --platform dy --lt qrcode --type search
Traceback (most recent call last):
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/main.py", line 66, in
asyncio.get_event_loop().run_until_complete(main())
File "/opt/homebrew/Cellar/python@3.10/3.10.16/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/main.py", line 56, in main
await crawler.start()
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/core.py", line 77, in start
await self.search()
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/core.py", line 106, in search
posts_res = await self.dy_client.search_info_by_keyword(keyword=keyword,
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/client.py", line 177, in search_info_by_keyword
return await self.get("/aweme/v1/web/general/search/single/", query_params, headers=headers)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/client.py", line 112, in get
await self.__process_req_params(uri, params, headers)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/client.py", line 91, in __process_req_params
a_bogus = await get_a_bogus(uri, query_string, post_data, headers["User-Agent"], self.playwright_page)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/help.py", line 51, in get_a_bogus
return get_a_bogus_from_js(url, params, user_agent)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/media_platform/douyin/help.py", line 67, in get_a_bogus_from_js
return douyin_sign_obj.call(sign_js_name, params, user_agent)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/_abstract_runtime_context.py", line 37, in call
return self._call(name, *args)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/_external_runtime.py", line 92, in _call
return self._eval("{identifier}.apply(this, {args})".format(identifier=identifier, args=args))
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/_external_runtime.py", line 78, in eval
return self.exec
(code)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/abstract_runtime_context.py", line 18, in exec
return self.exec(source)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/_external_runtime.py", line 85, in exec
output = self._exec_with_tempfile(source)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/_external_runtime.py", line 127, in _exec_with_tempfile
self._fail_on_non_zero_status(ret, stdoutdata, stderrdata)
File "/Users/peteyang/Desktop/Youtube Podcast/MediaCrawler/venv/lib/python3.10/site-packages/execjs/_external_runtime.py", line 134, in _fail_on_non_zero_status
raise ProcessExitedWithNonZeroStatus(status=status, stdout=stdoutdata, stderr=stderrdata)
execjs._exceptions.ProcessExitedWithNonZeroStatus: (1, '', 'The operation couldn’t be completed. Unable to locate a Java Runtime.\nPlease visit http://www.java.com for information on installing Java.\n\n')

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported command and trace the failure from main.py through media_platform/douyin/help.py, where execjs invokes the signing code. Check the Java runtime dependency indicated by the final error and verify the command completes without the ProcessExitedWithNonZeroStatus failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.