scrapinghub / scrapinghub/python-scrapinghub
Omitting `_key` hangs `_BatchWriter`
Open
Nobody has claimed this yet.
bug
- Dominant language
- Python
- Stars
- 205
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
Failing
# reprex.py
import os
import scrapinghub
store = (
scrapinghub.ScrapinghubClient(os.getenv('SH_APIKEY'))
.get_project(1234567890)
.collections.get_store("ok_to_mess_around_with")
)
writer = store.create_writer()
writer.write({"foo": "bar"})
print("write")
writer.write({"fizz": "buzz"})
print("write")
writer.flush()
print("flush")
python reprex.py
write
write
^CTraceback (most recent call last):
...
KeyboardInterrupt
Passing
import os
import scrapinghub
store = (
scrapinghub.ScrapinghubClient(os.getenv('SH_APIKEY'))
.get_project(1234567890)
.collections.get_store("ok_to_mess_around_with")
)
writer = store.create_writer()
writer.write({"_key": "foo", "foo": "bar"})
print("write")
writer.write({"_key": "fizz", "fizz": "buzz"})
print("write")
writer.flush()
print("flush")
python reprex.py
write
write
flush
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the provided reprex.py against a test store, then trace store.create_writer() into the _BatchWriter implementation. Compare the behavior of writes with and without _key; done means writer.flush() completes for records that omit _key, with the existing passing case still working.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100