apple / apple/containerization

LinuxPod post-create addContainer fails with "hotplug not supported" on 0.33.4; is there a public HotplugProvider extension to enable?

Open
#767 1 comment 0 reactions 1 assignee Claimed by @jglogan View on GitHub
Dominant language
Swift
Stars
8.9k
Forks
359
Avg merge
2d 20h
Merged PRs (30d)
13

Description

### Summary

I'm testing `LinuxPod` post-create `addContainer` on `apple/containerization`
`0.33.4` after PR #740 added hotplug interfaces:

https://github.com/apple/containerization/pull/740

The API surface exposes `LinuxPod.Configuration.extensions`,
`VMConfiguration.extensions`, `VZInstanceExtension`, and
`VZVirtualMachineInstance.hotplugProvider`, but I cannot find a public
extension/provider implementation to install. A live-pod post-create
`addContainer` attempt fails with:

```text
unsupported: "hotplug not supported"
```

### Environment

- Package: `apple/containerization`
- Version: `0.33.4`
- Checkout commit: `9275f365dd555c8f072e7d250d809f5eb7bdd746`
- macOS: 26.5.1
- Host architecture: arm64
- VM init image: `ghcr.io/apple/containerization/vminit:0.33.4`

### What I tried

1. Create a `LinuxPod` with `VZVirtualMachineManager`.
2. Add/register an initial container before `pod.create()`.
3. Start the initial container successfully.
4. Reuse the same live pod object.
5. Attempt `pod.addContainer(...)` after `pod.create()` using an ext4 block
rootfs mount.

The attempted post-create rootfs is a block mount:

```text
rootfsMountType=block
rootfsMountFormat=ext4
rootfsMountIsBlock=true
```

Runtime introspection around VM creation shows:

```text
vmConfigExtensionCount=0
vmConfigExtensionTypes=[]
hotplugProviderInstalled=false
hotplugProviderStatus=missing
```

Source inspection suggests `VZVirtualMachineInstance.hotplug(_:, id:)` throws
`ContainerizationError(.unsupported, message: "hotplug not supported")` when
`hotplugProvider` is nil.

### Question

Is there a public `VZInstanceExtension` / `HotplugProvider` implementation or
configuration step that consumers should append to
`LinuxPod.Configuration.extensions` to enable block-device and virtiofs hotplug
for post-create `addContainer`?

If the provider is not public yet, should `LinuxPod.addContainer` post-create be
considered unsupported for public `0.33.4` consumers, despite the public API
surface?

### Expected behavior

Either:

- a documented public extension/provider can be added to `LinuxPod.Configuration`
so post-create `addContainer` can hotplug an ext4 block rootfs into a running
VZ-backed pod; or
- the public API/docs clearly state that post-create `LinuxPod.addContainer`
requires a provider implementation that is not currently included by default.

### Actual behavior

`pod.addContainer(...)` after `pod.create()` fails with:

```text
unsupported: "hotplug not supported"
```

No provider appears to be installed by the default `VZVirtualMachineManager`
path, and I could not find a public provider/extension implementation in the
`0.33.4` source checkout.

### Follow-up: custom provider feasibility probe

I ran a second local probe to separate `no provider installed by default` from
`can a consumer install a provider and receive the post-create hotplug request?`

Result: a consumer-installed `VZInstanceExtension` / `HotplugProvider` can be
installed and `LinuxPod.addContainer(...)` after `pod.create()` does reach that
provider. However, I still could not complete a real LinuxPod container hotplug
for an ext4/block rootfs using only public APIs.

Summary:

```text
containerizationVersion=0.33.4
containerizationRevision=9275f365dd555c8f072e7d250d809f5eb7bdd746
customExtensionInstalled=true
hotplugProviderInstalled=true
providerHotplugCalled=true
publicUSBMassStorageAttachDetachWorks=true
postCreateAddContainerSucceeded=false
secondContainerStarted=false
realHotplugSucceeded=false
blocker=unsupportedRootfsBlockHotplug
```

I intentionally did not return a fake `AttachedFilesystem` or guess a `/dev/...`
path. I only counted hotplug as successful if the second container actually
started, which it did not.

Updated question:

Is there an intended public mechanism for a consumer-provided `HotplugProvider`
to attach an ext4/block rootfs image and return a LinuxPod-compatible
`AttachedFilesystem` source?

More specifically:

1. Is block/ext4 rootfs hotplug expected to be supported by the public VZ
backend?
2. If yes, what public API should a provider use to obtain the correct guest
block-device path?
3. Is the intended provider expected to use USB mass storage, Virtio block, NBD,
or another mechanism?
4. If this is not currently supported, should post-create
`LinuxPod.addContainer` be treated as unsupported for public VZ consumers
until a provider implementation is included?

For my local-dev orchestrator use case, this means I should not rely on
post-create hotplug yet. I am moving the product path toward fast pod recreate
plus rootfs copy avoidance / APFS clone / writable-layer investigation, while
keeping LinuxPod hotplug as an upstream/research track.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.