Azure / Azure/azure-cli

Unable to pass space-separated values with leading dashes

Open
#18,869 4 comments 2 reactions 1 assignee Claimed by @jiasli View on GitHub
argparse Discussion feature-request
Dominant language
Python
Stars
4.6k
Forks
3.5k
Avg merge
3d 2h
Merged PRs (30d)
60

Description

## Requirement

To run `az synapse spark job submit`, the user wants to pass `--instance`, `trainer` as values to `--arguments`.

Expected JSON:

```json
"arguments": [
"--instance",
"trainer"
]
```

## Symptom

`--arguments` is defined as

https://github.com/Azure/azure-cli/blob/39c91741b46de99e136e0d5438c4dbe4d672f08e/src/azure-cli/azure/cli/command_modules/synapse/_params.py#L383

So the user tries to pass `--instance`, `trainer` separated by spaces, and the command fails

```sh
# Line breaks for legibility
az synapse spark job submit --name foo --workspace-name personalizer-trainer --spark-pool-name persbackend --main-definition-file foo --main-class-name org.personalizer.backend.PipelineManager --executors 4 --executor-size Medium
--arguments --instance trainer
argument --arguments: expected at least one argument
```

As discussed in https://github.com/Azure/azure-cli/issues/16044#issuecomment-733955065, it is possible to use an equal sign `=` to concatenate the argument name and value for single-value argument:

```
az keyvault secret set -n mysecret --vault-name mykeyvault --value=-secret
```

> This usage is not documented by `argparse`: https://docs.python.org/3/library/argparse.html, https://docs.python.org/3/howto/argparse.html

However, it doesn't work for multi-value arguments:

```sh
# Line breaks for legibility
az synapse spark job submit --name foo --workspace-name personalizer-trainer --spark-pool-name persbackend --main-definition-file foo --main-class-name org.personalizer.backend.PipelineManager --executors 4 --executor-size Medium
--arguments=--instance trainer
unrecognized arguments: trainer
```

## Workarounds

There are several workaround worth considering:

1. Instead of using a `nargs='+'`, `--arguments` can take **single value** and split it with [`shlex.split`](https://docs.python.org/3/library/shlex.html#shlex.split)

2. Use `nargs=argparse.REMAINDER` to make `--arguments` take all remaining arguments. `azdev` uses this approach for [`--pytest-args`](https://github.com/Azure/azure-cli-dev-tools/blob/ac651115bdb66db69aea8c66c381fa24803f3486/azdev/params.py#L53) and perhaps this is the best way to go.

However, this usage has been removed from `argparse` document since Python 3.9 (https://bugs.python.org/issue17050):

> Since this feature is buggy, and there isn't an easy fix, we should probably remove any mention of it from the docs. We can still leave it as an undocumented legacy feature.

3. Replace `--arguments` with the `--` convention of `argparse` to mark all remaining arguments as **positional arguments** (https://docs.python.org/3/library/argparse.html#arguments-containing)

4. Add some special handling in `--arguments` so that dashes can be escaped, like `__instance`, or `^--instance`. But using characters like `` ` ``, `\` may cause other trouble with shell's interpretation.

No matter which workaround we choose, it is not an easy fix.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.