Datastore extension for Bonobo
- Dominant language
- No language data
- Stars
- 39
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
[Bonobo](https://www.bonobo-project.org/) is data processing toolkit for building ETL graphs in Python.
It would be really neat if there were a Datastore extension, in a similar vein to the [opendatasoft extension](https://github.com/python-bonobo/bonobo/blob/0.2/bonobo/ext/opendatasoft.py) so that users build pipelines like
```python
#from https://github.com/python-bonobo/bonobo/blob/0.2/bonobo/examples/datasets/coffeeshops.py
from os.path import dirname, realpath, join
import bonobo
from bonobo.ext.opendatasoft import OpenDataSoftAPI
OUTPUT_FILENAME = realpath(join(dirname(__file__), 'coffeeshops.txt'))
graph = bonobo.Graph(
OpenDataSoftAPI(dataset='liste-des-cafes-a-un-euro', netloc='opendata.paris.fr'),
lambda row: '{nom_du_cafe}, {adresse}, {arrondissement} Paris, France'.format(**row),
bonobo.FileWriter(path=OUTPUT_FILENAME),
)
if __name__ == '__main__':
bonobo.run(graph)
print('Import done, read {} for results.'.format(OUTPUT_FILENAME))
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading bonobo/ext/opendatasoft.py and the referenced datasets/coffeeshops.py example to understand the existing extension and pipeline shape. Determine the intended Datastore extension entry point and usage from those references; done should be a documented extension that supports the proposed datastore-backed pipeline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100