codeforboston / codeforboston/maple
Add MGL Hierarchy Descriptors to Bills
- Dominant language
- TypeScript
- Stars
- 56
- Forks
- 175
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 13
Description
## Problem
Bills often reference sections of the Massachusetts General Laws (MGL) in their text. We need to be able to easily access those sections to enable our LLM work (the model we use needs both the bill text and the relevant section texts to generate summaries), and can potentially use this data going forward to enrich the bill detail page.
## Success Criteria
* Documents in the `bills` collection in Firebase should have an additional field `mglSections` with `part`, `title`, `chapter`, and `section` subfields - this should contain all references within the bill text to existing sections of the MGL
* This field should be generated from the bill text when we scrape bills
* The LLM team currently uses ad-hoc scraper of the online MGL to get at this - their logic can likely be copied here.
* This data is generally available via the MA Legislature API under `GeneralLawParts`, `GeneralLawChapters`, and `GeneralLawSections`
* We should have unit tests verifying the correctness of the scraping logic
## Related Links
* Swagger API for Mass General Law: https://malegislature.gov/api/swagger/index.html?url=/api/swagger/v1/swagger.json#/
* LLM Scraper: https://github.com/maple-testimony/ml-maple-bill-summarization/blob/52fcff460570a1486c4f011261486c8ebe430791/demoapp/extract_mgl_sections.py#L138
* Bill Scraper: https://github.com/codeforboston/maple/blob/main/functions/src/bills/bills.ts
* Mass General Laws: https://malegislature.gov/Laws/GeneralLaws
## Other Info
* Reminder: Chapters run sequentially through all of the MGL - even across parts and titles. For example, Part II begins on Chapter 183. Many bills will not state the Part or Title (which are hierarchically above the Chapter), but only the Chapter and Section. But we can key that Chapter 1-182 are in Part 1; Chapter 183-210 are in Part 2; etc.
Contributor guide
Assessment
This issue has not been assessed yet.