apache / apache/cloudstack

Unable to launch a vm with a template which is direct download template and having nfs metalink

Open
#13,917 1 comment 0 reactions 0 assignees View on GitHub
bug component:templates
Dominant language
Java
Stars
3.1k
Forks
1.4k
Avg merge
6d 19h
Merged PRs (30d)
32

Description

### problem

Unable to launch a vm with a template which is direct download template and having nfs metalink

### versions

ACS 4.22.1

### The steps to reproduce the bug

1. Create a metalink file

Example
```



10485760

nfs://10.0.35.201/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2


```

2. Expose it via http server

python3 -m http.server 8081 --bind 0.0.0.0

3. Register it a direct download template in cloudstack

4. Launch a vm

5. Exception in agent.log

```

2026-08-19 04:36:53,965 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Request:Seq 1-6884877930342650195: { Cmd , MgmtId: 32985734710148, via: 1, Ver: v1, Flags: 100011, [{"org.apache.cloudstack.agent.directdownload.MetalinkDirectDownloadCommand":{"url":"http://10.0.35.201:8081/good-nfs.metalink","templateId":"202","destData":{"origUrl":"http://10.0.35.201:8081/good-nfs.metalink","uuid":"e50a9618-71f4-4b68-881a-54a2527d3e02","id":"202","format":"QCOW2","accountId":"2","hvm":"true","displayText":"test-nfs-metalink","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"24518d40-97cd-3433-bafa-dcb0291c03f9","name":"ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","id":"1","poolType":"NetworkFilesystem","host":"10.0.32.4","path":"/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","port":"2049","url":"NetworkFilesystem://10.0.32.4/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1/?ROLE=Primary&STOREUUID=24518d40-97cd-3433-bafa-dcb0291c03f9","isManaged":"false"}},"name":"202-2-7772abb0-fd86-32ff-add7-ef855cd70293","size":"(6580590.5000 TB) 7235435666033759572","hypervisorType":"KVM","bootable":"false","uniqueName":"202-2-7772abb0-fd86-32ff-add7-ef855cd70293","directDownload":"true","deployAsIs":"false","followRedirects":"false"},"destPool":{"uuid":"24518d40-97cd-3433-bafa-dcb0291c03f9","name":"ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","id":"1","poolType":"NetworkFilesystem","host":"10.0.32.4","path":"/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","port":"2049","url":"NetworkFilesystem://10.0.32.4/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1/?ROLE=Primary&STOREUUID=24518d40-97cd-3433-bafa-dcb0291c03f9","isManaged":"false"},"headers":{},"connectTimeout":"5000","soTimeout":"5000","templateSize":"(6580590.5000 TB) 7235435666033759572","format":"QCOW2","followRedirects":"false","wait":"10800","bypassHostMaintenance":"false"}}] }
2026-08-19 04:36:53,965 DEBUG [cloud.agent.Agent] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Processing command: org.apache.cloudstack.agent.directdownload.MetalinkDirectDownloadCommand

2026-08-19 04:36:53,965 DEBUG [storage.resource.StorageSubsystemCommandHandlerBase] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Executing command MetalinkDirectDownloadCommand: [{"url":"http://10.0.35.201:8081/good-nfs.metalink","templateId":202,"destData":{"origUrl":"http://10.0.35.201:8081/good-nfs.metalink","uuid":"e50a9618-71f4-4b68-881a-54a2527d3e02","id":202,"format":"QCOW2","accountId":2,"hvm":true,"displayText":"test-nfs-metalink","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"24518d40-97cd-3433-bafa-dcb0291c03f9","name":"ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","id":1,"poolType":"NetworkFilesystem","host":"10.0.32.4","path":"/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","port":2049,"url":"NetworkFilesystem://10.0.32.4/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1/?ROLE=Primary&STOREUUID=24518d40-97cd-3433-bafa-dcb0291c03f9","isManaged":false}},"name":"202-2-7772abb0-fd86-32ff-add7-ef855cd70293","size":7235435666033759572,"hypervisorType":"KVM","bootable":false,"uniqueName":"202-2-7772abb0-fd86-32ff-add7-ef855cd70293","directDownload":true,"deployAsIs":false,"followRedirects":false},"destPool":{"uuid":"24518d40-97cd-3433-bafa-dcb0291c03f9","name":"ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","id":1,"poolType":"NetworkFilesystem","host":"10.0.32.4","path":"/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1","port":2049,"url":"NetworkFilesystem://10.0.32.4/acs/primary/ref-trl-12221-k-Mol8-kiran-chavala/ref-trl-12221-k-Mol8-kiran-chavala-kvm-pri1/?ROLE=Primary&STOREUUID=24518d40-97cd-3433-bafa-dcb0291c03f9","isManaged":false},"headers":{},"connectTimeout":5000,"soTimeout":5000,"templateSize":7235435666033759572,"format":"QCOW2","followRedirects":false,"wait":10800,"bypassHostMaintenance":false}].

2026-08-19 04:36:53,965 DEBUG [kvm.storage.KVMStorageProcessor] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Verifying temporary location for downloading the template exists on the host
2026-08-19 04:36:53,967 DEBUG [kvm.storage.KVMStorageProcessor] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Checking for free space on the host for downloading the template with physical size: 413 and virtual size: 7235435666033759572
2026-08-19 04:36:53,967 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Executing command [/bin/bash -c df --output=avail /var/lib/libvirt/images -B 1 | tail -1 ].
2026-08-19 04:36:53,973 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Successfully executed process [68470] for command [/bin/bash -c df --output=avail /var/lib/libvirt/images -B 1 | tail -1 ].
2026-08-19 04:36:53,973 INFO [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Trying to fetch storage pool 24518d40-97cd-3433-bafa-dcb0291c03f9 from libvirt
2026-08-19 04:36:53,974 DEBUG [kvm.resource.LibvirtConnection] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Looking for libvirtd connection at: qemu:///system
2026-08-19 04:36:53,978 DEBUG [kvm.storage.LibvirtStorageAdaptor] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Successfully refreshed pool 24518d40-97cd-3433-bafa-dcb0291c03f9 Capacity: (3.4170 TB) 3757022642176 Used: (2.7297 TB) 3001299238912 Available: (703.82 GB) 755723403264
2026-08-19 04:36:53,987 ERROR [cloud.utils.UriUtils] (AgentRequest-Handler-3:[]) (logid:fe7c1979) no hash tag in XML response...
2026-08-19 04:36:53,987 INFO [direct.download.MetalinkDirectTemplateDownloader] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Metalink downloader created, metalink URL: http://10.0.35.201:8081/good-nfs.metalink parsed - 1 urls and 0 checksums found
2026-08-19 04:36:53,987 DEBUG [kvm.storage.KVMStorageProcessor] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Trying to download template
2026-08-19 04:36:53,987 INFO [direct.download.MetalinkDirectTemplateDownloader] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Trying to download Template from URL: nfs://10.0.35.201/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2
2026-08-19 04:36:53,987 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Executing command [/bin/bash -c mount -t nfs 10.0.35.201:/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ].
2026-08-19 04:36:53,999 WARN [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Execution of process [68473] for command [/bin/bash -c mount -t nfs 10.0.35.201:/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ] failed.
2026-08-19 04:36:53,999 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Exit value of process [68473] for command [/bin/bash -c mount -t nfs 10.0.35.201:/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ] is [32].
2026-08-19 04:36:53,999 WARN [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Process [68473] for command [/bin/bash -c mount -t nfs 10.0.35.201:/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ] encountered the error: [mount.nfs: mount point /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 does not exist].
2026-08-19 04:36:53,999 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Executing command [/bin/bash -c cp /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 null/null/template/n/null/96fa2884-a398-48f9-acb6-7d00e894520f.qcow2 ].
2026-08-19 04:36:54,005 WARN [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Execution of process [68475] for command [/bin/bash -c cp /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 null/null/template/n/null/96fa2884-a398-48f9-acb6-7d00e894520f.qcow2 ] failed.
2026-08-19 04:36:54,006 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Exit value of process [68475] for command [/bin/bash -c cp /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 null/null/template/n/null/96fa2884-a398-48f9-acb6-7d00e894520f.qcow2 ] is [1].
2026-08-19 04:36:54,006 WARN [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Process [68475] for command [/bin/bash -c cp /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2 null/null/template/n/null/96fa2884-a398-48f9-acb6-7d00e894520f.qcow2 ] encountered the error: [cp: cannot stat '/mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2': No such file or directory].
2026-08-19 04:36:54,006 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Executing command [/bin/bash -c umount /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ].
2026-08-19 04:36:54,011 WARN [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Execution of process [68476] for command [/bin/bash -c umount /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ] failed.
2026-08-19 04:36:54,011 DEBUG [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Exit value of process [68476] for command [/bin/bash -c umount /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ] is [32].
2026-08-19 04:36:54,011 WARN [utils.script.Script] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Process [68476] for command [/bin/bash -c umount /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04 ] encountered the error: [umount: /mnt/e460cef8-0a57-4080-878f-ddcb1abf4f04: no mount point specified.].
2026-08-19 04:36:54,011 INFO [direct.download.MetalinkDirectTemplateDownloader] (AgentRequest-Handler-3:[]) (logid:fe7c1979) Successfully downloaded Template from URL: nfs://10.0.35.201/export/templates/ubuntu24-cloud-image-root-password-with-tools.qcow2
```

### What to do about it?

Should be able to successfully launch a vm from metalink url which has a nfs file

https://docs.cloudstack.apache.org/en/4.22.1.0/adminguide/templates/_bypass-secondary-storage-kvm.html

Contributor guide

Open the contributing guide

Research direction

Start with the KVMStorageProcessor and MetalinkDirectTemplateDownloader entry points shown in the agent.log trace, then reproduce the NFS metalink launch using the XML and steps in the issue. Trace the mount, copy, and cleanup results; done means a direct-download template with an NFS metalink launches a VM successfully without the reported mount-point and false-success errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.