apple / apple/container

[Bug]: virtio-fs: disk space from deleted files in a bind-mounted volume is not released until the container is stopped

Open
#2,021 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
49.9k
Forks
1.8k
Avg merge
1d 20h
Merged PRs (30d)
22

Description

### I have done the following

- [x] I have searched the existing issues
- [ ] If possible, I've reproduced the issue using the 'main' branch of this project

### Steps to reproduce

**Check existing available space on host**
```shell
$ df -h ~
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk3s1 460Gi 387Gi 48Gi 90% 1.7M 500M 0% /System/Volumes/Data
```

**Create a data directory and start a simple container**
```shell
$ mkdir -p repro/data
$ container run -d --rm --name repro -v repro/data:/data alpine:latest sleep infinity
```

**Generate a 10GB file on the host and delete it within the container**
```shell
$ dd if=/dev/urandom of=repro/data/file.bin bs=1M count=10240
$ container exec repro rm /data/file.bin
```

**Check free space on host**
```shell
$ df -h ~
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk3s1 460Gi 397Gi 38Gi 92% 1.7M 395M 0% /System/Volumes/Data
```

Note storage has dropped by 10Gi, despite the file having been deleted.

**Check the `repro/data` directory**
```shell
$ ls -lah repro/data
```

**Check `lsof` for open handlers**
```shell
$ sudo lsof +L1 | grep repro/data
com.apple 41792 cpressland 17r REG 1,13 10737418240 0 58800424 /Users/cpressland/repro/data/file.bin
```

**Stop the container and check free space**
```shell
$ container stop repro
repro
$ df -h ~
Filesystem Size Used Avail Capacity iused ifree %iused Mounted on
/dev/disk3s1 460Gi 387Gi 48Gi 90% 1.7M 499M 0% /System/Volumes/Data
```

### Problem description

When a file inside a container's bind-mounted (-v) host directory is deleted from within the container, the underlying disk blocks are not released back to the host filesystem. `du`, Finder, and Grand Perspective all show no trace of the file, the directory entry is genuinely gone but the space stays allocated. `lsof +L1` on the host confirms the container's own VM process (`/System/Library/Frameworks/Virtualization.framework/Versions/A/X`) still holds an open handle on the deleted inode. The space is only reclaimed when the container is stopped.

**Expected behavior**
Once rm removes the last directory reference to a file and no process retains an open handle to it, the disk space should be released back to the host filesystem.

**Actual behavior**
`df -h` shows no space recovered. `sudo lsof +L1` reveals the container's Virtualization.framework VM process itself still holding a read handle on the deleted files:
```
com.apple 41792 cpressland 17r REG 1,13 10737418240 0 58800424 /Users/cpressland/repro/data/file.bin
```

Space is only reclaimed when the container is stopped (`container stop`), which tears down the VM and its virtio-fs device with it.

**Key observations**
* The read is irrelevant. `file.bin` was created directly on the host and never opened by any process inside the guest, only `rm`'d via `container exec`. It leaked identically to files the guest had fully read. This points at the delete/unlink handling in the virtio-fs implementation rather than read-side caching.
* File size is irrelevant across the range tested (10KB–10GB).
* It does not self-heal. Confirmed in a real production workload that space from files deleted at 2am was still held nearly 12 hours later, with no other activity in between.
* This is easy to hit in practice. Any container that writes to a bind-mounted directory and later has that content deleted or moved out (a download client, a build cache, a log rotator) will silently consume its entire backing volume over time, while every host-side inspection tool reports normal usage.

### Environment

```markdown
- OS: macOS 26.5.2 (25F84)
- Xcode: N/A
- Container: container CLI version 1.1.0 (build: release, commit: 5973b9c)
```

### Code of Conduct

- [x] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the shown container run, exec rm, df, and lsof commands on macOS. Then trace the virtio-fs delete/unlink handling described in the report. Done means deleting a file from a bind-mounted volume releases its host disk space without stopping the container, with coverage for the reported behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, swift
Domain
infrastructure, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.