codeforboston / codeforboston/maple

Legislator Profile - Campaign Finance Data Pipeline

Open
#2,131 0 comments 0 reactions 1 assignee Claimed by @J-C-L View on GitHub
backend Ready for Development scraper
Dominant language
TypeScript
Stars
56
Forks
175
Avg merge
2d 5h
Merged PRs (30d)
13

Description

## Summary

As part of the Legislator Profile project, we want to make the Campaign Finance data (made publicly available via OCPF) associated with a Legislator visible to our users through the new Legislator Profile page. We don't currently scrape Campaign Finance data, so we'll need to add a new scraper that plumbs this data into our system to make it available for the Front End.

## Success Criteria
- [ ] New scraper that checks OCPF regularly for new data and plumbs any new data into our system (likely a new firestore collection e.g. `campaignFinance`)
- [ ] This will need to be queryable by Legislator (Ideally by the Member Code - since that's what the Legislator Profile is keyed off - so we may need to do a little work here to map legislators to members depending on the format of the OCPF data)
- [ ] Depending on the volume/format of the data dump from OCPF, it may be worthwhile to pre-compute the key statistics here by legislator (e.g. total raised, total spent, small donors, and breakdown by industry) and save the summaries separately.
- [ ] The Legislator Profile Frontend will (for now) only display the most recent election cycle's data
- [ ] One-off script to backfill historical data into the new collection (ideally going back to ~2009/2010 - the earliest year we're likely support since that's when the MA Legislature API came out)

## Screenshots

Image

## Additional Links
- Mockup: https://maple-leg-profiles.netlify.app/
- OCPF: https://www.ocpf.us/Data/Downloads

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.