Unable to download ISO images from Microsoft servers
- Dominant language
- Go
- Stars
- 15.8k
- Forks
- 3.3k
- PR merge metrics
- PR metrics pending
Description
#### Overview of the Issue
Packer fails to fetch ISO images specified via the `iso_url` property from Microsoft servers--even when the URLs function in other utilities (e.g. web browsers and cURL).
Microsoft's download server appears to enforce strict requirements on the sequence of query string parameters. In the process of adding the `checksum` parameter, Packer re-orders the query string parameters.
Here are the combinations I tried with cURL (credentials redacted) along with the HTTP status code returned by software-download.microsoft.com
https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?t=TTT&e=EEE&h=HHH 200
https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?t=TTT&h=HHH&e=EEE 403
https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?e=EEE&t=TTT&h=HHH 403
https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?e=EEE&h=HHH&t=TTT 403
https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?h=HHH&t=TTT&e=EEE 500
https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?h=HHH&e=EEE&t=TTT 500
#### Reproduction Steps
1. Visit https://www.microsoft.com/en-us/software-download/windows10ISO
2. Select the "Windows 10 (multi-edition)" option
3. Choose the English language
4. Create a Packer configuration file which includes a build whose `iso_url` property is the URL provided by Microsoft
Here's a Bash script that probes all URL combinations, given valid credentials
```bash
#!/bin/bash
set -e
base='https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso'
# Insert valid credentials for the following three values:
t=""
e=""
h=""
vars="t
e
h"
for first in $vars; do
for second in $vars; do
if [ "$first" = "$second" ] ; then
continue
fi
for third in $vars; do
if [ "$first" = "$third" -o "$second" = "$third" ] ; then
continue
fi
url="$base?$first=${!first}&$second=${!second}&$third=${!third}"
result=$(
curl \
--silent \
--head \
--output /dev/null \
--write-out '%{http_code}\n' \
$url
)
echo $url $result
done
done
done
```
### Packer version
$ packer --version
1.7.10
### Simplified Packer Template
```json
{
"builders": [
{
"type": "virtualbox-iso",
"guest_os_type": "Windows10_64",
"iso_checksum": "sha256:7F6538F0EB33C30F0A5CBBF2F39973D4C8DEA0D64F69BD18E406012F17A8234F",
"iso_url": "https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?t=TTT&e=EEE&h=HHH",
"ssh_username": "packer",
"ssh_password": "packer"
}
]
}
```
### Operating system and Environment details
$ uname -a
Linux bruce 4.15.0-167-generic #175-Ubuntu SMP Wed Jan 5 01:56:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
### Log Fragments and crash.log files
(credentials redacted for clarity)
log
$ PACKER_LOG=1 packer build --debug tiny.json
2022/02/14 20:08:02 [INFO] Packer version: 1.7.10 [go1.17.6 linux amd64]
2022/02/14 20:08:02 Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 [TRACE] discovering plugins in /usr/local/bin
2022/02/14 20:08:02 [DEBUG] Discovered plugin: arm = /usr/local/bin/packer-builder-arm
2022/02/14 20:08:02 [INFO] using external builders: [arm]
2022/02/14 20:08:02 Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 [TRACE] discovering plugins in /home/mike/.packer.d/plugins
2022/02/14 20:08:02 [TRACE] discovering plugins in .
2022/02/14 20:08:02 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/02/14 20:08:02 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/mike/.packerconfig
2022/02/14 20:08:02 [WARN] Config file doesn't exist: /home/mike/.packerconfig
2022/02/14 20:08:02 Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 [INFO] Setting cache directory: /home/mike/.cache/packer
2022/02/14 20:08:02 Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 [TRACE] Starting internal plugin packer-builder-virtualbox-iso
2022/02/14 20:08:02 Starting plugin: /usr/local/bin/packer-1.7.10 []string{"/usr/local/bin/packer-1.7.10", "plugin", "packer-builder-virtualbox-iso"}
2022/02/14 20:08:02 Waiting for RPC address for: /usr/local/bin/packer-1.7.10
2022/02/14 20:08:02 packer-1.7.10 plugin: [INFO] Packer version: 1.7.10 [go1.17.6 linux amd64]
2022/02/14 20:08:02 packer-1.7.10 plugin: Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 packer-1.7.10 plugin: [INFO] PACKER_CONFIG env var not set; checking the default config file path
2022/02/14 20:08:02 packer-1.7.10 plugin: [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/mike/.packerconfig
2022/02/14 20:08:02 packer-1.7.10 plugin: [WARN] Config file doesn't exist: /home/mike/.packerconfig
2022/02/14 20:08:02 packer-1.7.10 plugin: Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 packer-1.7.10 plugin: [INFO] Setting cache directory: /home/mike/.cache/packer
2022/02/14 20:08:02 packer-1.7.10 plugin: Old default config directory found: /home/mike/.packer.d
2022/02/14 20:08:02 packer-1.7.10 plugin: args: []string{"packer-builder-virtualbox-iso"}
2022/02/14 20:08:02 packer-1.7.10 plugin: Plugin address: unix /tmp/packer-plugin3939556317
2022/02/14 20:08:02 Received unix RPC address for /usr/local/bin/packer-1.7.10: addr is /tmp/packer-plugin3939556317
2022/02/14 20:08:02 packer-1.7.10 plugin: Waiting for connection...
2022/02/14 20:08:02 packer-1.7.10 plugin: Serving a plugin connection...
2022/02/14 20:08:02 Preparing build: virtualbox-iso
A shutdown_command was not specified. Without a shutdown command, Packer
will forcibly halt the virtual machine, which may result in data loss.
2022/02/14 20:08:02 Build debug mode: true
2022/02/14 20:08:02 Force build: false
2022/02/14 20:08:02 On error:
2022/02/14 20:08:02 Debug enabled, so waiting for build to finish: virtualbox-iso
2022/02/14 20:08:02 Starting build run: virtualbox-iso
2022/02/14 20:08:02 Running builder: virtualbox-iso
2022/02/14 20:08:02 [INFO] (telemetry) Starting builder virtualbox-iso
Warning: Warning when preparing build: "virtualbox-iso"
A shutdown_command was not specified. Without a shutdown command, Packer
will forcibly halt the virtual machine, which may result in data loss.
Debug mode enabled. Builds will not be parallelized.
virtualbox-iso: output will be in this color.
2022/02/14 20:08:02 packer-1.7.10 plugin: VBoxManage path: /usr/bin/VBoxManage
2022/02/14 20:08:02 packer-1.7.10 plugin: VBoxManage --version output: 6.1.32r149290
2022/02/14 20:08:02 packer-1.7.10 plugin: VirtualBox version: 6.1.32
2022/02/14 20:08:02 packer-1.7.10 plugin: guest_additions_url is blank; querying driver for iso.
2022/02/14 20:08:02 packer-1.7.10 plugin: Found Default Guest Additions ISO: /usr/share/virtualbox/VBoxGuestAdditions.iso
2022/02/14 20:08:02 packer-1.7.10 plugin: Guest additions URL: /usr/share/virtualbox/VBoxGuestAdditions.iso
2022/02/14 20:08:02 packer-1.7.10 plugin: Acquiring lock for: /usr/share/virtualbox/VBoxGuestAdditions.iso (/home/mike/.cache/packer/26fb5ce50fc4b178e2f2ab9965e7b69240ae8239.iso.lock)
==> virtualbox-iso: Retrieving Guest additions
2022/02/14 20:08:02 packer-1.7.10 plugin: Leaving retrieve loop for Guest additions
==> virtualbox-iso: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: Trying /usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: /usr/share/virtualbox/VBoxGuestAdditions.iso => /usr/share/virtualbox/VBoxGuestAdditions.iso
==> virtualbox-iso: Pausing after run of step 'StepDownloadGuestAdditions'. Press enter to continue.
==> virtualbox-iso: Retrieving ISO
==> virtualbox-iso: Trying https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?t=TTT&e=EEE&h=HHH
==> virtualbox-iso: Trying https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?checksum=sha256%3A7f6538f0eb33c30f0a5cbbf2f39973d4c8dea0d64f69bd18e406012f17a8234f&e=EEE&h=HHH&t=TTT
2022/02/14 20:08:06 packer-1.7.10 plugin: Acquiring lock for: https://software-download.microsoft.com/pr/Win10_21H2_English_x64.iso?checksum=sha256%3A7f6538f0eb33c30f0a5cbbf2f39973d4c8dea0d64f69bd18e406012f17a8234f&e=EEE&h=HHH&t=TTT (/home/mike/.cache/packer/6fde7f4443177ff77e5b881f269f5da10d194c7e.iso.lock)
==> virtualbox-iso: Download failed bad response code: 403
==> virtualbox-iso: error downloading ISO: [bad response code: 403]
2022/02/14 20:08:06 packer-1.7.10 plugin: Leaving retrieve loop for ISO
==> virtualbox-iso: Pausing before cleanup of step 'StepDownloadGuestAdditions'. Press enter to continue.
2022/02/14 20:08:50 [INFO] (telemetry) ending virtualbox-iso
Build 'virtualbox-iso' errored after 47 seconds 430 milliseconds: error downloading ISO: [bad response code: 403]
2022/02/14 20:08:50 Waiting on builds to complete...
==> Wait completed after 47 seconds 430 milliseconds
==> Wait completed after 47 seconds 430 milliseconds
2022/02/14 20:08:50 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2022/02/14 20:08:50 machine readable: virtualbox-iso,error []string{"error downloading ISO: [bad response code: 403]"}
==> Builds finished but no artifacts were created.
2022/02/14 20:08:50 [INFO] (telemetry) Finalizing.
==> Some builds didn't complete successfully and had errors:
--> virtualbox-iso: error downloading ISO: [bad response code: 403]
==> Builds finished but no artifacts were created.
2022/02/14 20:08:50 waiting for all plugin processes to complete...
2022/02/14 20:08:50 /usr/local/bin/packer-1.7.10: plugin process exited
Contributor guide
Assessment
This issue has not been assessed yet.