Unable to pass space-separated values with leading dashes
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
## Requirement
To run `az synapse spark job submit`, the user wants to pass `--instance`, `trainer` as values to `--arguments`.
Expected JSON:
```json
"arguments": [
"--instance",
"trainer"
]
```
## Symptom
`--arguments` is defined as
https://github.com/Azure/azure-cli/blob/39c91741b46de99e136e0d5438c4dbe4d672f08e/src/azure-cli/azure/cli/command_modules/synapse/_params.py#L383
So the user tries to pass `--instance`, `trainer` separated by spaces, and the command fails
```sh
# Line breaks for legibility
az synapse spark job submit --name foo --workspace-name personalizer-trainer --spark-pool-name persbackend --main-definition-file foo --main-class-name org.personalizer.backend.PipelineManager --executors 4 --executor-size Medium
--arguments --instance trainer
argument --arguments: expected at least one argument
```
As discussed in https://github.com/Azure/azure-cli/issues/16044#issuecomment-733955065, it is possible to use an equal sign `=` to concatenate the argument name and value for single-value argument:
```
az keyvault secret set -n mysecret --vault-name mykeyvault --value=-secret
```
> This usage is not documented by `argparse`: https://docs.python.org/3/library/argparse.html, https://docs.python.org/3/howto/argparse.html
However, it doesn't work for multi-value arguments:
```sh
# Line breaks for legibility
az synapse spark job submit --name foo --workspace-name personalizer-trainer --spark-pool-name persbackend --main-definition-file foo --main-class-name org.personalizer.backend.PipelineManager --executors 4 --executor-size Medium
--arguments=--instance trainer
unrecognized arguments: trainer
```
## Workarounds
There are several workaround worth considering:
1. Instead of using a `nargs='+'`, `--arguments` can take **single value** and split it with [`shlex.split`](https://docs.python.org/3/library/shlex.html#shlex.split)
2. Use `nargs=argparse.REMAINDER` to make `--arguments` take all remaining arguments. `azdev` uses this approach for [`--pytest-args`](https://github.com/Azure/azure-cli-dev-tools/blob/ac651115bdb66db69aea8c66c381fa24803f3486/azdev/params.py#L53) and perhaps this is the best way to go.
However, this usage has been removed from `argparse` document since Python 3.9 (https://bugs.python.org/issue17050):
> Since this feature is buggy, and there isn't an easy fix, we should probably remove any mention of it from the docs. We can still leave it as an undocumented legacy feature.
3. Replace `--arguments` with the `--` convention of `argparse` to mark all remaining arguments as **positional arguments** (https://docs.python.org/3/library/argparse.html#arguments-containing)
4. Add some special handling in `--arguments` so that dashes can be escaped, like `__instance`, or `^--instance`. But using characters like `` ` ``, `\` may cause other trouble with shell's interpretation.
No matter which workaround we choose, it is not an easy fix.
Contributor guide
Assessment
This issue has not been assessed yet.