Azure / Azure/azure-storage-azcopy

AzCopy downloads with --from-to BlobPipe fails when downloading multiple files

Open
#2,575 4 comments 0 reactions 1 assignee Claimed by @pranavmalik-msft View on GitHub
feature request
Dominant language
Go
Stars
698
Forks
284
Avg merge
3d 6h
Merged PRs (30d)
11

Description

### Which version of the AzCopy was used?
azcopy version 10.23.0

### Which platform are you using? (ex: Windows, Mac, Linux)
Linux x86-64

### What command did you run?

``` sh
# Downloading multiple files to local dir works
$ azcopy cp "https://ACCOUNT.blob.core.windows.net/CONTAINER/*?SAS" --include-pattern="*.csv.gz" .
100.0 %, 3 Done, 0 Failed, 0 Pending, 0 Skipped, 3 Total,
...

# Downloading multiple files to pipe fails silently with exitcode 0
$ azcopy cp "https://ACCOUNT.blob.core.windows.net/CONTAINER/*?SAS" --include-pattern="*.csv.gz" --from-to BlobPipe | pv > /dev/null
0,00 B 0:00:01 [0,00 B/s]
$ azcopy cp "https://ACCOUNT.blob.core.windows.net/CONTAINER/*?SAS" --include-pattern="*.csv.gz" --from-to BlobPipe > testfiles
$ echo $?
0
$ du testfiles
0 testfiles

# Downloading single file to pipe works
$ azcopy cp "https://ACCOUNT.blob.core.windows.net/CONTAINER/data01.csv.gz?SAS" --include-pattern="*.csv.gz" --from-to BlobPipe | pv > /dev/null
345MiB 0:01:25 [4,03MiB/s]
```

### What problem was encountered?
I expect azcopy to be able to download multiple files to a BlobPipe, it does not work.

### How can we reproduce the problem in the simplest way?
Try to download multiple files from a storage account with `--from-to BlobPipe`

### Have you found a mitigation/solution?
Not yet, probably possible to do it in multiple steps, list files first and then start multiple azcopy processes..

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.