Reporting job status failed with error code: INVALID_ARGUMENT: Dataflow GCP
Open
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
I am fetching data from PostgreSQL first and creating a P Collection but unfortunately job is getting failed when rows count becomes greater than 250.
I am getting this error when running on Dataflow (GCP). On local it is running as expected.
Contributor guide
Research direction
Start by comparing the PostgreSQL-to-P Collection pipeline when run locally and on Dataflow (GCP), then inspect the Dataflow logs around the INVALID_ARGUMENT failure. Reproduce the issue with more than 250 rows and identify the missing configuration or behavior needed for the Dataflow job to complete successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- gcp, postgresql
- Domain
- cloud, data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100