CouncilDataProject / CouncilDataProject/cdp-scrapers

Scrapers provide API to return specific information outside of get_events()

Open
#59 3 comments 0 reactions 1 assignee Claimed by @dphoria View on GitHub
enhancement
Dominant language
Python
Stars
27
Forks
18
PR merge metrics
No merged PRs in 30d

Description

### Feature Description

_A clear and concise description of the feature you're requesting._

`cdp-backend` must call `get_events()` to get _any_ ingestion model data. We want to be able to e.g. pull just `Persons`, or `Matters`.

### Use Case

When a new council member is elected, the pipeline must pull events spanning some specific date(s), get the entire `List[EventIngestionModel]` data set, get at the `Vote.person` or `Matter.sponsor` to obtain information for just this one new `Person`.

This new APIs would allow to do this in a single (or at least fewer) steps.

### Solution

_Please describe your ideal solution._

```Python
def get_all_persons() -> List[Person]:
"""
return all Persons
"""
return []

def get_updated_information(target: IngestionModel) -> IngestionModel:
"""
Return input IngestionModel updated with new information if exists

Parameters
----------
target: IngestionModel
e.g. Person, Matter

Returns
-------
IngestionModel
input target updated with new information, or returned as-is
"""
return target
```

### Alternatives

_Please describe any alternatives you've considered, even if you've dismissed them._

```Python
def get_all_persons() -> List[Person]:
# same as above
return []

def get_updated_matter(matter: Matter) -> Matter:
# like the proposed get_updated_information() but simpler just for Matter
return matter
```

This issue is continuing the conversation started in https://github.com/CouncilDataProject/cdp-scrapers/pull/58#pullrequestreview-847167881.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.