CouncilDataProject / CouncilDataProject/cdp-data

Parallelize mass conversion of transcripts to CSV

Open
#18 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Jupyter Notebook
Stars
5
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Currently, we use `thread_map` to do a lot of data fetching for getting events, sessions, transcripts, videos, etc. However our `store_transcript_as_csv` logic is just a loop that takes a long time. It would be preferable to use `process_map` (or w.e. the function name is) in a similar fashion as all the other processing we do to parallelize this process.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.