microsoft / microsoft/aspire

Docker Compose: project resources cannot write to their own persistent volume (no fsGroup equivalent)

Open
#19,422 0 comments 0 reactions 0 assignees View on GitHub
area-integrations
Dominant language
C#
Stars
6.3k
Forks
991
Avg merge
2d 15h
Merged PRs (30d)
196

Description

## Summary

`WithVolume(name, target, env)` works correctly on Kubernetes but is effectively unusable for **project/executable** resources deployed to Docker Compose: the deployed app gets `UnauthorizedAccessException` on every write to its own persistent volume.

This was found by an E2E test added in #19404 (`DockerComposeDeployWithVolumeTests`). The generated Compose artifacts are correct — the failure is purely at runtime, so unit/snapshot tests cannot catch it.

## Repro

Deploy a project resource with a volume to Compose:

```csharp
builder.AddProject("server")
.WithVolume("serverdata", "/data", env: "DATA_PATH");
```

The generated `docker-compose.yaml` is exactly right:

```yaml
services:
server:
environment:
DATA_PATH: "/data"
volumes:
- type: "volume"
target: "/data"
source: "serverdata"
read_only: false
volumes:
serverdata:
driver: "local"
```

But every request that writes to `$DATA_PATH` fails:

```
System.UnauthorizedAccessException: Access to the path '/data/marker.txt' is denied.
---> System.IO.IOException: Permission denied
```

## Root cause

.NET container images run as a non-root user, but a fresh Docker named volume mounted at a path that does not exist in the image is created `root:root 0755`:

```
image USER: 1654
container: uid=1654(app) gid=1654(app)
/data: drwxr-xr-x 2 0 0
```

Kubernetes does not have this problem because `KubernetesResource.cs` deliberately handles it:

```csharp
if (hasPersistentVolumeBinding)
{
// fsGroup is a supplemental group, not the image's primary GID. A stable
// publisher-owned value lets non-root images access supported volumes without
// coupling the manifest to image-specific identities.
var securityContext = Workload.PodTemplate.Spec.SecurityContext ??= new();
securityContext.FsGroup ??= DefaultPersistentVolumeFsGroup; // 2000
securityContext.FsGroupChangePolicy ??= DefaultPersistentVolumeFsGroupChangePolicy; // OnRootMismatch
}
```

The Compose publisher has no equivalent, so the two compute environments diverge on the feature's flagship scenario.

## Docker Compose has no `fsGroup`

There is no direct equivalent in Docker or Compose. I evaluated the conventional workarounds empirically (Docker 29.4.3):

| Pattern | Fresh volume | **Pre-existing** volume | Image change | App runs as root |
|---|---|---|---|---|
| `RUN mkdir && chown` in the image | ✅ WRITE OK | ❌ **WRITE DENIED** | yes | no |
| Entrypoint `chown` | ✅ | ✅ | yes | yes |
| `user: "0:0"` | ✅ | ✅ | no | yes |
| `group_add` alone | ❌ | ❌ **WRITE DENIED** | no | no |

The commonly recommended image-side fix relies on Docker copying image directory ownership into an *empty* volume. It silently stops working once the volume has content — the worst failure mode for a persistence feature.

## Proposed fix: a faithful `fsGroup` analogue

`fsGroup` group-owns the volume, sets `g+rwx` + setgid, and adds the GID as a supplementary group. All three have Compose equivalents:

```
init service (root): chgrp -R 2000 /data && chmod -R g+rwX /data && chmod g+s /data
app service: group_add: ["2000"]
depends_on: { : { condition: service_completed_successfully } }
```

Verified against the hard case — a pre-existing root-owned volume containing data:

```
before init: drwxr-xr-x 0 0 → WRITE DENIED
after init: drwxrwsr-x 0 2000 → WRITE OK
```

The setgid bit makes new files inherit the group, matching Kubernetes behaviour. The app stays non-root; only the short-lived init service runs as root.

Notes:
- Reuse the existing `DefaultPersistentVolumeFsGroup = 2000` constant so both environments agree.
- `fsGroupChangePolicy = OnRootMismatch` maps to making the init idempotent (skip the `chgrp` when the top-level group already matches), which also avoids the slow-chown-on-large-volumes problem.
- **No model plumbing needed** — `Service.GroupAdd` and `Service.DependsOn` / `ServiceDependency.Condition` already exist in `Aspire.Hosting.Docker`. This is generator logic only.
- One shared init service per Compose environment is probably tidier than one per volume-bound resource.

## Test coverage

`DockerComposeDeployWithVolumeTests` (added in #19404) currently asserts the generated Compose shape and that the running container has the env var and a named volume mounted at that path. The data-round-trip assertion (write → force-recreate container → read back) was deliberately left out because it cannot pass until this is fixed. That assertion should be added here once the fix lands — it is the part that actually proves durability.

Contributor guide

Open the contributing guide

Research direction

Compare the Compose publisher with KubernetesResource.cs, then inspect DockerComposeDeployWithVolumeTests and the existing Service.GroupAdd and Service.DependsOn/ServiceDependency.Condition model support. Implement the generator-side volume initialization flow using the existing persistent-volume group constant, and add the data round-trip assertion that writes, force-recreates the container, and reads the data back.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, docker, docker-compose
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.