azcopy creates directory on download failure which triggers further issue

Bug report

Based on comment: https://github.com/nextflow-io/nextflow/pull/5799#issuecomment-2931082656

Due to code in link it will then try to generate a folder that also crashs. It took a bit of time to find this bug :)

Expected behavior and actual behavior

We can probably be more intelligent about the difference between downloading a directory or a path here.

The current situation:

1. azcopy tries to download the input files
2. It fails, then creates a directory
3. It then tries to download a directory
4. It also fails

This obfuscates the failure in step 2 making it hard to debug.

Instead, we could handle downloading paths or directories better. This will probably require some investigation and/or an update to azcopy to see the behaviour but I think it should be possible.

Steps to reproduce the problem

TBC! But it should be possible by:

1. Set process.memory to 1GB
2. Try to download a large (>50GB) file
3. Watch the task on Azure Batch
4. Check the working directory.

Program output

🤷

Environment

• Nextflow version: [?]
• Java version: [?]
• Operating system: [macOS, Linux, etc]
• Bash version: (use the command $SHELL --version)

Additional context

Originally described by @KH-NN, any additional context would be helpful.

Start with plugins/nf-azure/src/test/nextflow/executor/BashWrapperBuilderWithAzTest.groovy around lines 84-90 and review linked pull request #6187. Investigate the AzCopy download behavior for files versus directories and use the reproduction steps when available; done means a failed file download does not create a misleading directory or obscure the original failure.