catalyst-cooperative / catalyst-cooperative/mozilla-sec-eia

Integrate record linkage into production

Open
#119 0 comments 0 reactions 1 assignee Claimed by @katie-lamb View on GitHub
Dominant language
Jupyter Notebook
Stars
2
Forks
0
PR merge metrics
No merged PRs in 30d

Description

### Overview

I've settled on splink model parameters in a notebook for linking SEC to EIA. Now it's time to get this working in our pipeline and create an output table that's usable by the masses.

### Success Criteria

How will we know that we're done?

* [ ] Record linkage of SEC to EIA runs in production
* [ ] Output table of SEC filers and subsidiaries links to EIA
* [ ] Validation metrics are logged in `MLflow`

```[tasklist]
### Next steps
- [x] Create a module that formats the SEC table into a denormalized, flattened output table
- [x] Create a preprocessing module for EIA
- [ ] Create a record linkage module that runs the splink notebook and integrate the notebook into Dagster
- [ ] Add a column that links the SEC table to EIA utilities
- [ ] Make a PUDL name cleaner improvements PR
- [ ] Integrate MLflow to log model metrics
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.