apple / apple/containerization
LinuxPod post-create addContainer fails with "hotplug not supported" on 0.33.4; is there a public HotplugProvider extension to enable?
- Dominant language
- Swift
- Stars
- 8.9k
- Forks
- 359
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 13
Description
### Summary
I'm testing `LinuxPod` post-create `addContainer` on `apple/containerization`
`0.33.4` after PR #740 added hotplug interfaces:
https://github.com/apple/containerization/pull/740
The API surface exposes `LinuxPod.Configuration.extensions`,
`VMConfiguration.extensions`, `VZInstanceExtension`, and
`VZVirtualMachineInstance.hotplugProvider`, but I cannot find a public
extension/provider implementation to install. A live-pod post-create
`addContainer` attempt fails with:
```text
unsupported: "hotplug not supported"
```
### Environment
- Package: `apple/containerization`
- Version: `0.33.4`
- Checkout commit: `9275f365dd555c8f072e7d250d809f5eb7bdd746`
- macOS: 26.5.1
- Host architecture: arm64
- VM init image: `ghcr.io/apple/containerization/vminit:0.33.4`
### What I tried
1. Create a `LinuxPod` with `VZVirtualMachineManager`.
2. Add/register an initial container before `pod.create()`.
3. Start the initial container successfully.
4. Reuse the same live pod object.
5. Attempt `pod.addContainer(...)` after `pod.create()` using an ext4 block
rootfs mount.
The attempted post-create rootfs is a block mount:
```text
rootfsMountType=block
rootfsMountFormat=ext4
rootfsMountIsBlock=true
```
Runtime introspection around VM creation shows:
```text
vmConfigExtensionCount=0
vmConfigExtensionTypes=[]
hotplugProviderInstalled=false
hotplugProviderStatus=missing
```
Source inspection suggests `VZVirtualMachineInstance.hotplug(_:, id:)` throws
`ContainerizationError(.unsupported, message: "hotplug not supported")` when
`hotplugProvider` is nil.
### Question
Is there a public `VZInstanceExtension` / `HotplugProvider` implementation or
configuration step that consumers should append to
`LinuxPod.Configuration.extensions` to enable block-device and virtiofs hotplug
for post-create `addContainer`?
If the provider is not public yet, should `LinuxPod.addContainer` post-create be
considered unsupported for public `0.33.4` consumers, despite the public API
surface?
### Expected behavior
Either:
- a documented public extension/provider can be added to `LinuxPod.Configuration`
so post-create `addContainer` can hotplug an ext4 block rootfs into a running
VZ-backed pod; or
- the public API/docs clearly state that post-create `LinuxPod.addContainer`
requires a provider implementation that is not currently included by default.
### Actual behavior
`pod.addContainer(...)` after `pod.create()` fails with:
```text
unsupported: "hotplug not supported"
```
No provider appears to be installed by the default `VZVirtualMachineManager`
path, and I could not find a public provider/extension implementation in the
`0.33.4` source checkout.
### Follow-up: custom provider feasibility probe
I ran a second local probe to separate `no provider installed by default` from
`can a consumer install a provider and receive the post-create hotplug request?`
Result: a consumer-installed `VZInstanceExtension` / `HotplugProvider` can be
installed and `LinuxPod.addContainer(...)` after `pod.create()` does reach that
provider. However, I still could not complete a real LinuxPod container hotplug
for an ext4/block rootfs using only public APIs.
Summary:
```text
containerizationVersion=0.33.4
containerizationRevision=9275f365dd555c8f072e7d250d809f5eb7bdd746
customExtensionInstalled=true
hotplugProviderInstalled=true
providerHotplugCalled=true
publicUSBMassStorageAttachDetachWorks=true
postCreateAddContainerSucceeded=false
secondContainerStarted=false
realHotplugSucceeded=false
blocker=unsupportedRootfsBlockHotplug
```
I intentionally did not return a fake `AttachedFilesystem` or guess a `/dev/...`
path. I only counted hotplug as successful if the second container actually
started, which it did not.
Updated question:
Is there an intended public mechanism for a consumer-provided `HotplugProvider`
to attach an ext4/block rootfs image and return a LinuxPod-compatible
`AttachedFilesystem` source?
More specifically:
1. Is block/ext4 rootfs hotplug expected to be supported by the public VZ
backend?
2. If yes, what public API should a provider use to obtain the correct guest
block-device path?
3. Is the intended provider expected to use USB mass storage, Virtio block, NBD,
or another mechanism?
4. If this is not currently supported, should post-create
`LinuxPod.addContainer` be treated as unsupported for public VZ consumers
until a provider implementation is included?
For my local-dev orchestrator use case, this means I should not rely on
post-create hotplug yet. I am moving the product path toward fast pod recreate
plus rootfs copy avoidance / APFS clone / writable-layer investigation, while
keeping LinuxPod hotplug as an upstream/research track.
Contributor guide
Assessment
This issue has not been assessed yet.