GoogleCloudPlatform / GoogleCloudPlatform/training-data-analyst

Notebook: Data Preprocessing for Machine Learning

Open
#449 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
8.6k
Forks
6.1k
Avg merge
4h 44m
Merged PRs (30d)
2

Description

Hey,

I took samples from your code in order to test **Dataflow**. When trying to run the pipeline on the full-dataset, your examples notebook says: "Note, you can change the first arugment to "None" to process the full dataset." This did not work for me. I had to change the `create_query`-fct:
Instead of
`AND MOD(ABS(FARM_FINGERPRINT(CAST(pickup_datetime AS STRING))), EVERY_N) = 1`
I had to put
`AND MOD(ABS(FARM_FINGERPRINT(CAST(pickup_datetime AS STRING))), EVERY_N) = 0`
and then use in the function call EVERY_N=1:
`preprocess("1", "DataflowRunner")`

My notebook is available [here](https://github.com/tarrade/proj_DL_models_and_pipelines_with_GCP/blob/master/notebook/dataflow_example.ipynb).

Original Notebook "
Data Preprocessing for Machine Learning": [/courses/machine_learning/deepdive/04_advanced_preprocessing/a_dataflow.ipynb](https://github.com/GoogleCloudPlatform/training-data-analyst/blob/master/courses/machine_learning/deepdive/04_advanced_preprocessing/a_dataflow.ipynb)

Best, Henry

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.