`buildextend-virtualbox` and `buildextend-vmware` improperly handle raw disks >=8GB

Bug Report

The buildextend-virtualbox and buildextend-vmware formats generate OVA files. The OVA standard defines that it's a USTAR format tarball containing an OVF definition and the associated files (primarily the VMDK disk files). USTAR tarball format only supports individual files <8GB.

If the vmdk is 8GB or larger, the VMDK needs to be generated as a split disk so the files can be placed into the OVA (USTAR tarball), otherwise a build error is produced by the tar command. Currently this isn't done.

Environment

What operating system is being used to run coreos-assembler?

Container image on Fedora 39 and on Ubuntu 22.04.

What operating system is being assembled?

Customized Fedora CoreOS image.

Is coreos-assembler running in Podman or Docker?

Yes, either/both.

If Podman, is coreos-assembler running privileged or unprivileged?

Privileged.

Expected Behavior

The OVA file is generated successfully.

Actual Behavior

The OVA file is not generated, and a build error is reported from the tar command.
Example:

2024-05-01 17:43:54,748 INFO - Running command: ['qemu-img', 'convert', '-f', 'qcow2', '-O', 'vmdk', '/srv/tmp/tmp1lkujnoa/fedora-coreos-e37b7ac.20240501170000-qemu.x86_64.qcow2.working', '-o', 'subformat=streamOptimized', '/srv/tmp/tmp1lkujnoa/fedora-coreos-e37b7ac.20240501170000-virtualbox.x86_64.vmdk']
2024-05-01 17:49:08,455 INFO - Running command: ['qemu-img', 'info', '--output=json', '/srv/tmp/tmp1lkujnoa/fedora-coreos-e37b7ac.20240501170000-virtualbox.x86_64.vmdk']
2024-05-01 17:49:08,460 INFO - Processing work image callback
2024-05-01 17:49:08,494 INFO - Running command: ['qemu-img', 'info', '--output=json', '/srv/tmp/tmp1lkujnoa/fedora-coreos-e37b7ac.20240501170000-virtualbox.x86_64.vmdk']
2024-05-01 17:49:08,499 INFO - Running command: ['tar', '--owner=0', '--group=0', '-C', '/srv/tmp/tmp1lkujnoa', '-ch', '--format=ustar', '-f', '/srv/builds/e37b7ac.20240501170000/x86_64/fedora-coreos-e37b7ac.20240501170000-virtualbox.x86_64.ova', 'coreos.ovf', 'disk.vmdk']
tar: value 9055751168 out of off_t range 0..8589934591
tar: Exiting with failure status due to previous errors
2024-05-01 17:49:08,500 ERROR - Command returned bad exitcode
2024-05-01 17:49:08,500 ERROR - COMMAND: ['tar', '--owner=0', '--group=0', '-C', '/srv/tmp/tmp1lkujnoa', '-ch', '--format=ustar', '-f', '/srv/builds/e37b7ac.20240501170000/x86_64/fedora-coreos-e37b7ac.20240501170000-virtualbox.x86_64.ova', 'coreos.ovf', 'disk.vmdk']

Reproduction Steps

1. Using Fedora CoreOS config, run cosa init
2. Add a huge hard to compress file to the rootfs overrides dd if=/dev/urand of=overrides/rootfs/somebigfile.bin bs=4M count=4000 status=progress (16GB file of random binary data here)
3. Fetch & Build: cosa fetch && cosa build
4. Build vbox (or vmware): cosa buildextend-virtualbox or cosa buildextend-vmware
5. See the tar error

Other Information

According to the QEMU block driver documentation for VMDK format, it appears it's possible to add an extra subformat option twoGbMaxExtentSparse which the associated code suggests would generate split VMDK files with each file being only 2GB in size, and isn't mutually exclusive with the existing streamOptimized option (or the others non-subformat options used for the vmware build.

This could easily be added to the QemuVariant data structures in the code that define how to generate the output disk, and would get handled when the VMDK is created. The OVF template also wouldn't need to be changed at all for this, the split VMDK format produces a disk.vmdk file that then references all the split part files, so the way to refer to it in the OVF is the same regardless of whether it's a split VMDK or not.

What would need to be changed though is the files that need to be collected for inclusion in the OVA. Instead of it being only a single disk.vmdk and the generated OVF file, it would need to include the all the VMDK split part files as well.

Start with the QemuVariant data structures used by buildextend-virtualbox and buildextend-vmware, then reproduce the failure with the listed cosa init, fetch, build, and buildextend commands. Check how the generated disk files are collected for the OVA. Done means a large disk is split into files that fit USTAR and the OVA is generated successfully.