`-o tsv` emits CRLF on Windows, breaking `for x in $(az ... -o tsv)` shell loops with misleading 400 errors
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
### Describe the bug
On Windows, `az ... --output tsv` writes each row terminated by **CRLF** (`\r\n`) instead of LF (`\n`). When that output is consumed by a POSIX shell (Git Bash / MSYS2 / WSL-on-Windows-files / bash-from-PowerShell) via `for x in $(...)`, every token except the last carries a stray `\r`. That `\r` gets interpolated into the URL path of the next `az` call, and ARM responds with `HTTP 400 Bad Request`. The CLI's error message — `Operation returned an invalid status 'Bad Request'` — does not reveal the underlying cause.
Net effect: a common shell pattern shown in many Microsoft Learn tutorials silently fails on Windows for every iteration **except the last one** in the loop.
### To Reproduce
Any namespace/topic combination works. Example using Service Bus:
```bash
RG=
NS=
for topic in $(az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv); do
count=$(az servicebus topic subscription list --resource-group "$RG" --namespace-name "$NS" --topic-name "$topic" --query "length(@)" -o tsv 2>&1)
printf "%-40s %s\n" "$topic" "$count"
done
```
Actual output (14 topics, only the alphabetically last works):
```
topic1 ERROR: Operation returned an invalid status 'Bad Request'
topic2 ERROR: Operation returned an invalid status 'Bad Request'
```
### Environment summary
```
azure-cli 2.80.0
core 2.80.0
telemetry 1.1.0
Extensions:
account 0.2.5
authV2 1.0.1
azure-devops 1.0.2
redisenterprise 1.4.0
subscription 0.1.3
Dependencies:
msal 1.34.0b1
azure-mgmt-resource 23.3.0
Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
OS: Windows 11 Pro 10.0.26200
Shell: Git Bash (MSYS2 / mintty); also reproducible from PowerShell-invoked bash subshells
```
### Proof of root cause
`az ... -o tsv` writes CRLF row terminators on Windows:
```bash
The bash `$(...)` substitution then carries `\r` on every token except the last (bash strips a trailing `\n` and the `\r` just before it in that single edge case — the other tokens keep their `\r` because the `\r` precedes a non-terminal `\n`).
When the next `az` call receives `--topic-name "$topic"`, the inner CLI inserts the `\r` into the ARM REST URL path. ARM responds 400. The CLI surfaces only `Operation returned an invalid status 'Bad Request'` and does not echo the ARM error body, so the cause is invisible to the user.
### Workaround
Strip `\r` from the captured output:
```bash
for topic in $(az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv | tr -d '\r'); do
count=$(az servicebus topic subscription list --resource-group "$RG" --namespace-name "$NS" --topic-name "$topic" --query "length(@)" -o tsv)
printf "%-40s %s\n" "$topic" "$count"
done
```
### Proof of root cause
`az ... -o tsv` writes CRLF row terminators on Windows:
```bash
$ az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv | head -3 | od -c
0000000 t o p i c 1 \r \n t o p i c 2 \r \n
0000020 t o p i c 3 \r \n
0000030
```
After applying `| tr -d '\r'` the loop returns the expected counts for all 14 topics.
### Comparison of output formats on Windows
| Output mode | Line ending | Safe in shell `for` loops? |
|---|---|---|
| `-o tsv` | **CRLF** | **No — this bug** |
| `-o json` | LF | n/a (not row-delimited) |
| `-o jsonc` | LF | n/a |
| `-o yaml` | LF | n/a |
| `-o table` | CRLF (decorative) | n/a |
Only the **machine-parseable, row-delimited** format `tsv` is affected. `csv` likely has the same issue but I have not retested it for this report.
### Suggested fix
The CLI's `tsv` formatter inherits Python's default text-mode line translation on Windows. Two options:
1. **Reconfigure stdout for tsv/csv output** — at format-dispatch time, call `sys.stdout.reconfigure(newline='\n')` when the user selected `tsv` or `csv`. Smallest change, easiest to review.
2. **Write tsv as bytes** — `sys.stdout.buffer.write(line.encode() + b'\n')`. More invasive but eliminates any platform-specific newline handling.
Option 1 should be a one-line change in the format dispatch (`azure-cli-core/azure/cli/core/commands/__init__.py` or wherever the formatter lookup happens).
### Bonus suggestion (separate issue territory)
When the underlying ARM call returns 4xx, the CLI should print the ARM error body, not just `Operation returned an invalid status 'Bad Request'`. ARM almost always includes a `code` and `message` field that would have pointed at the bad URL character immediately. The current behavior of swallowing those fields turns a 30-second diagnosis into a 30-minute one.
### Why this matters
The pattern `for x in $(az ... -o tsv); do ...; done` appears verbatim in Microsoft Learn tutorials, Azure samples, and many blog posts. Every Windows user copying those snippets hits this bug and assumes their script, their credentials, their permissions, or their resource state is wrong. Few think "the CLI's tsv output has the wrong line endings." A one-line formatter fix removes a recurring scripting trap.
### Anything else
Confirmed not present in PowerShell — PowerShell splits `tsv` output on `[\r\n]+`, masking the bug. The issue is bash-on-Windows-specific in observed effect, but the **root cause** (CRLF emission by the CLI on Windows) is the CLI's responsibility, not bash's.
```
### Related command
```bash
RG=
NS=
for topic in $(az servicebus topic list --resource-group "$RG" --namespace-name "$NS" --query "[].name" -o tsv); do
count=$(az servicebus topic subscription list --resource-group "$RG" --namespace-name "$NS" --topic-name "$topic" --query "length(@)" -o tsv 2>&1)
printf "%-40s %s\n" "$topic" "$count"
done
### Errors
Actual output (14 topics, only the alphabetically last works):
```
topic1 ERROR: Operation returned an invalid status 'Bad Request'
topic2 ERROR: Operation returned an invalid status 'Bad Request'
### Issue script & Debug output
Actual output (14 topics, only the alphabetically last works):
```
topic1 ERROR: Operation returned an invalid status 'Bad Request'
topic2 ERROR: Operation returned an invalid status 'Bad Request'
### Expected behavior
### Comparison of output formats on Windows
| Output mode | Line ending | Safe in shell `for` loops? |
|---|---|---|
| `-o tsv` | **CRLF** | **No — this bug** |
| `-o json` | LF | n/a (not row-delimited) |
| `-o jsonc` | LF | n/a |
| `-o yaml` | LF | n/a |
| `-o table` | CRLF (decorative) | n/a |
Only the **machine-parseable, row-delimited** format `tsv` is affected. `csv` likely has the same issue but I have not retested it for this report.
### Environment Summary
```
azure-cli 2.80.0
core 2.80.0
telemetry 1.1.0
Extensions:
account 0.2.5
authV2 1.0.1
azure-devops 1.0.2
redisenterprise 1.4.0
subscription 0.1.3
Dependencies:
msal 1.34.0b1
azure-mgmt-resource 23.3.0
Python location 'C:\Program Files\Microsoft SDKs\Azure\CLI2\python.exe'
OS: Windows 11 Pro 10.0.26200
Shell: Git Bash (MSYS2 / mintty); also reproducible from PowerShell-invoked bash subshells
```
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.