GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst
Notebook: Data Preprocessing for Machine Learning
- Dominant language
- Jupyter Notebook
- Stars
- 8.6k
- Forks
- 6.1k
- Avg merge
- 4h 44m
- Merged PRs (30d)
- 2
Description
Hey,
I took samples from your code in order to test **Dataflow**. When trying to run the pipeline on the full-dataset, your examples notebook says: "Note, you can change the first arugment to "None" to process the full dataset." This did not work for me. I had to change the `create_query`-fct:
Instead of
`AND MOD(ABS(FARM_FINGERPRINT(CAST(pickup_datetime AS STRING))), EVERY_N) = 1`
I had to put
`AND MOD(ABS(FARM_FINGERPRINT(CAST(pickup_datetime AS STRING))), EVERY_N) = 0`
and then use in the function call EVERY_N=1:
`preprocess("1", "DataflowRunner")`
My notebook is available [here](https://github.com/tarrade/proj_DL_models_and_pipelines_with_GCP/blob/master/notebook/dataflow_example.ipynb).
Original Notebook "
Data Preprocessing for Machine Learning": [/courses/machine_learning/deepdive/04_advanced_preprocessing/a_dataflow.ipynb](https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/04_advanced_preprocessing/a_dataflow.ipynb)
Best, Henry
Contributor guide
Assessment
This issue has not been assessed yet.