codeforboston / codeforboston/clean-slate-data

Consolidate data pipeline scripts

Open
#180 1 comment 0 reactions 3 assignees Claimed by @agathaalmunir View on GitHub
low priority
Dominant language
Jupyter Notebook
Stars
28
Forks
13
PR merge metrics
No merged PRs in 30d

Description

]Create single notebook / script for the data flow from deidentified-but-still-raw data to ‘prosecution_charges_detailed’. Right now, prosecution_charges is both an input and output of two different scripts, without a clear indication of what should be run first. Would be helpful to just condense those steps into a single script.

- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/MA_Data-1_Raw.ipynb
- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/MA_Data-2_MergeCharges.ipynb
- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/MA_Data-2_MergeCharges_alt.ipynb (similar to the above but incorporates some of these two, and some additional edits to the expungeability):
-- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/new_expungability_info_join_emily.R
-- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/sex_murder_columns.R
- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/MA_Data_revised_joining.R
- https://github.com/codeforboston/clean-slate/blob/master/analyses/notebooks/Middlesex_Clean.ipynb

The general pipeline is in the readme in the /[notebooks ](https://github.com/codeforboston/clean-slate/tree/master/analyses/notebooks) page.

Thoughts on how to do this are also drafted here: [Procedure for adding new MA prosecution data](https://docs.google.com/document/d/16eEJ3fTVZiZyFMAZWTxdeLbbByLY_8FG60NanAfhYL8/edit?usp=sharing)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.