codeforboston / codeforboston/maple

Legislator Profile - Elections Data Pipeline

Open
#2,130 0 comments 0 reactions 1 assignee Claimed by @delexagon View on GitHub
backend Ready for Development scraper
Dominant language
TypeScript
Stars
56
Forks
175
Avg merge
2d 5h
Merged PRs (30d)
13

Description

## Summary

For the Legislator Profile project, we want to show past elections data for each legislator. We don't currently have that data in our backend, so we need to create a scraper to pull that data into our system to make it available for the front-end to display.

Election Results are available from the Secretary of State's website (e.g. https://electionstats.state.ma.us/elections/search/year_from:2022/year_to:2026/office_id:9/stage:General)

## Success Criteria
- [ ] New Scraper that runs daily to check the Secretary of State's website for new election results and pull them into our system if so
- [ ] This will likely demand a new firestore collection (e.g. `electionResults`)
- [ ] We (currently) only care about General Election results, not primaries
- [ ] This data updates very infrequently (outside special elections, each seat is only up once every two/four years).
- [ ] Create a script to run this scraper against historical data (going back to ~2009/2010 - the earliest court we're likely to support) once )
- [ ] Data before 2025 is unlikely to ever change - we can grab all historical data in one fell swoop, then only monitor for new data

## Additional Links
- Secretary of State Election Results Website: https://electionstats.state.ma.us/elections/search/year_from:2022/year_to:2026/office_id:9/stage:General
- Mockup: https://maple-leg-profiles.netlify.app/

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.