scrapinghub / scrapinghub/python-scrapinghub

KeyError: 'status' when trying to schedule spider

Open
#156 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
205
Forks
62
PR merge metrics
No merged PRs in 30d

Description

I am getting this error when trying to schedule a spider. This is happening with version 2.3.1

Traceback (most recent call last):
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/legacy.py", line 157, in _decode_response
    if data['status'] == 'ok':
KeyError: 'status'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/client/exceptions.py", line 69, in wrapped
    return method(*args, **kwargs)
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/client/__init__.py", line 19, in _request
    return super(Connection, self)._request(*args, **kwargs)
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/legacy.py", line 143, in _request
    return self._decode_response(response, format, raw)
  File "/home/molveyra/.local/share/virtualenvs/mollie-AtuAN_AE/lib/python3.8/site-packages/scrapinghub/legacy.py", line 169, in _decode_response
    raise APIError("JSON response does not contain status")
scrapinghub.legacy.APIError: JSON response does not contain status

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read scrapinghub/legacy.py, especially _request and _decode_response, then reproduce spider scheduling with version 2.3.1. Inspect the JSON response that lacks status and trace how it is produced; done means the scheduling failure is explained and the error path is covered by the repository's existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.