CouncilDataProject / CouncilDataProject/cdp-backend

Make event gather pipeline dry-runnable

Open
#85 0 comments 0 reactions 0 assignees View on GitHub
enhancement event gather pipeline
Dominant language
Python
Stars
25
Forks
28
PR merge metrics
No merged PRs in 30d

Description

### Feature Description

_A clear and concise description of the feature you're requesting._

Add a parameter to the pipeline that enables us to dryrun it without any database or file store upload.

### Use Case

_Please provide a use case to help us understand your request in context._

Would be very useful to dryrun the pipeline during deployment bot configuration validation.

### Solution

_Please describe your ideal solution._

* wait for #57 to allow passing a single event or multiple events to the pipeline directly
* if the event doesnt come with captions, mock the transcript generation interaction
* mock all file store and database upload interactions
* add a task to the pipeline that runs _if_ it was a dry run that stores the created database models to a single json file. (this should be possible because `fireo` has a `to_dict` method on all model objects.)

### Alternatives

_Please describe any alternatives you've considered, even if you've dismissed them._

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.