apache / apache/beam

[Bug]: Flags with similar name as Pipeline options collide when passed with single dash

Open
#25,943 0 comments 0 reactions 1 assignee Claimed by @riteshghorse View on GitHub
bug python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

We have seen a behavior for Python SDK using interactive runner starting 2.42.0 that when a flag which is same as another pipeline option is passed to dataflow runner with single dash, it escapes the valid argument check before job submission and creates problem at runtime. Moreover, the error is logged as info message and the job fails after an hour because of worker stuck error.

For example, passing `--f=abc` which is a valid shorthand for `--flexrs_goal` does the argument validation before job submission. But when `-f=abc` is used, it escapes this check and logs an info message
![image](https://user-images.githubusercontent.com/25881114/227221880-7f1c0d38-b397-428e-ba21-fd0a1b8e8dc4.png)
on dataflow.

### Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

### Issue Components

- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.