CouncilDataProject / CouncilDataProject/cdp-scrapers
Scrapers provide API to return specific information outside of get_events()
- Dominant language
- Python
- Stars
- 27
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
### Feature Description
_A clear and concise description of the feature you're requesting._
`cdp-backend` must call `get_events()` to get _any_ ingestion model data. We want to be able to e.g. pull just `Persons`, or `Matters`.
### Use Case
When a new council member is elected, the pipeline must pull events spanning some specific date(s), get the entire `List[EventIngestionModel]` data set, get at the `Vote.person` or `Matter.sponsor` to obtain information for just this one new `Person`.
This new APIs would allow to do this in a single (or at least fewer) steps.
### Solution
_Please describe your ideal solution._
```Python
def get_all_persons() -> List[Person]:
"""
return all Persons
"""
return []
def get_updated_information(target: IngestionModel) -> IngestionModel:
"""
Return input IngestionModel updated with new information if exists
Parameters
----------
target: IngestionModel
e.g. Person, Matter
Returns
-------
IngestionModel
input target updated with new information, or returned as-is
"""
return target
```
### Alternatives
_Please describe any alternatives you've considered, even if you've dismissed them._
```Python
def get_all_persons() -> List[Person]:
# same as above
return []
def get_updated_matter(matter: Matter) -> Matter:
# like the proposed get_updated_information() but simpler just for Matter
return matter
```
This issue is continuing the conversation started in https://github.com/CouncilDataProject/cdp-scrapers/pull/58#pullrequestreview-847167881.
Contributor guide
Assessment
This issue has not been assessed yet.