hasadna / hasadna/knesset-data-python

votes html scraper should not fail silently by default

Open
#4 14 comments 0 reactions 0 assignees View on GitHub
noob friendly
Dominant language
Python
Stars
2
Forks
14
PR merge metrics
No merged PRs in 30d

Description

following PR #3 - it looks like in case an unknown member is encountered it will fail silently

in knesset-data-python we would like to be as low-level as possible and make as little assumptions as possible about how the data will be used

in this case, silently ignoring an error might cause unexpected problems down the line

I prefer that by default we will raise an exception, and add a skip_exceptions parameter which will instead of raising an exception yield the exception object as part of the data stream - to allow the end-user decide how to handle the exception

this is done in the dataservice scrapers, see for example this unit test -
https://github.com/hasadna/knesset-data-python/blob/master/knesset_data/dataservice/tests/base/test_exceptions.py#L38

Contributor guide

Open the contributing guide

Research direction

Start with the votes HTML scraper and compare its exception handling with the dataservice scraper behavior described in PR #3. Read knesset_data/dataservice/tests/base/test_exceptions.py around line 38, then verify that unknown members raise by default and that skip_exceptions yields the exception object when enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.