canonical / canonical/workshop
LXD socket tunnel (`tunnel` interface) broken since 0.9.4: `lxc` inside a workshop fails with "connection reset by peer"
- Dominant language
- Go
- Stars
- 108
- Forks
- 17
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 40
Description
**Component:** workshop snap, `lxd-tunnel` SDK / `tunnel` interface
**Severity:** High — `lxc` (and any LXD client) inside a workshop is completely unusable
**Affected versions:** workshop **0.9.4** and later (regression)
**Introduced by:** commit `dbbca6369325e3d6cb0cee2db090ba7009909f1f` — "Drop privileges for tunnels from workshop to host" (2026-07-09)
## Environment
- Host: Ubuntu 26.04 LTS, kernel `7.0.0-28-generic` (x86_64)
- workshop: snap **0.9.4-ebb8bb7** (rev 386, `latest/stable`, `classic`)
- LXD: snap **6.9** (host daemon at `/var/snap/lxd/common/lxd/unix.socket`)
- Host user: uid 10000, **primary** gid 10000; `lxd` group gid 114, membership **supplementary only** (`getent group lxd` → `lxd:x:114:myuser`)
## Summary
Since workshop 0.9.4, a workshop that tunnels the host LXD socket into the container exposes the socket, but **every connection is reset**. Inside the container:
```
$ lxc list
Error: LXD unix socket "/var/lib/workshop/run/host-lxd/unix.socket" not accessible: Get "http://unix.socket/1.0": read unix @->/var/lib/workshop/run/host-lxd/unix.socket: read: connection reset by peer
```
`curl` through the same tunneled socket fails identically:
```
$ curl --unix-socket /var/lib/workshop/run/host-lxd/unix.socket http://unix.socket/1.0
curl: (56) Recv failure: Connection reset by peer
```
Direct access to the real host socket works fine:
```
$ curl --unix-socket /var/snap/lxd/common/lxd/unix.socket http://unix.socket/1.0
{"api_status":"stable", ...}
```
## Steps to reproduce
### 1. Define a workshop with a `tunnel` interface to the host LXD socket
`workshop.yaml` (abridged — only the tunnel-relevant parts):
```yaml
name: dev
base: ubuntu@24.04
sdks:
- name: system
slots:
lxd:
interface: tunnel
endpoint: /var/snap/lxd/common/lxd/unix.socket
# ... other SDKs (go, node, rust, opencode, vscode-remote ...)
connections:
- plug: lxd-tunnel:lxd
slot: system:lxd
```
Where `lxd-tunnel` is a project SDK whose plug attaches the tunnel to the container:
`.workshop/lxd-tunnel/sdk.yaml`:
```yaml
name: lxd-tunnel
plugs:
lxd:
interface: tunnel
endpoint: /var/lib/workshop/run/host-lxd/unix.socket
```
### 2. Launch the workshop
```
$ workshop launch dev
```
### 3. Try to use LXD from inside the container
```
$ workshop exec dev -- lxc list
```
**Expected:** list of host LXD instances/projects.
**Actual:**
```
Error: LXD unix socket "/var/lib/workshop/run/host-lxd/unix.socket" not accessible: Get "http://unix.socket/1.0": read unix @->/var/lib/workshop/run/host-lxd/unix.socket: read: connection reset by peer
```
The failure is on the *first* request — any LXD client (Go SDK, `curl`) is affected.
## Root cause
The tunnel proxy device created for the workshop carries the host user's credentials as `security.uid`/`security.gid`:
```
$ lxc --project workshop.10000 profile show dev-6bc2a0cf-lxd-tunnel
devices:
lxd-tunnel_lxd:
bind: instance
connect: unix:/var/snap/lxd/common/lxd/unix.socket
gid: "1000"
listen: unix:/var/lib/workshop/run/host-lxd/unix.socket
security.gid: "10000"
security.uid: "10000"
type: proxy
uid: "1000"
```
LXD spawns `forkproxy` for the *connect* side of this `bind: instance` proxy. Observed on the host, that process runs as the host user with **no supplementary groups**:
```
$ ps -o user,pid,args -p
didier.+ ... /snap/lxd/current/sbin/lxd forkproxy -- ... unix:/var/lib/workshop/run/host-lxd/unix.socket ... unix:/var/snap/lxd/common/lxd/unix.socket 1000 1000 0644 10000 10000
$ grep -E "Uid|Gid|Groups" /proc//status
Uid: 10000 10000 10000 10000
Gid: 10000 10000 10000 10000
Groups:
```
The host LXD socket is owned `srw-rw---- root:lxd` (0660):
```
$ ls -l /var/snap/lxd/common/lxd/unix.socket
srw-rw---- 1 root lxd 0 Aug 3 16:52 /var/snap/lxd/common/lxd/unix.socket
```
For this user, `lxd` access is a **supplementary** group. Once `forkproxy`'s gid is the user's **primary** gid (10000), the supplementary `lxd` membership is gone, the connect to the 0660 `root:lxd` socket fails with `EACCES`, `forkproxy` aborts and closes the server-side connection, and the client — which has already written its request — sees `connection reset by peer`.
This is a direct consequence of the 0.9.4 hardening change, which adds exactly these two keys to every workshop→host tunnel device:
```go
// internal/workshop/lxd/lxd_backend.go (proxyToLxdDevice)
case workshop.WorkshopToHost:
device["bind"] = "instance"
device["security.uid"] = usr.Uid
device["security.gid"] = usr.Gid
```
```go
// internal/interfaces/lxd_device/spec.go (addProxyEntry)
device["security.uid"] = s.User.Uid
device["security.gid"] = s.User.Gid
```
`usr`/`s.User` come from `os/user.Lookup` (`internal/osutil/user.go`), i.e. the *primary* uid/gid from `/etc/passwd`.
### Why it affects only some setups
- **Broken:** host user whose **primary** gid is *not* the `lxd` group gid — i.e. anyone added to `lxd` with `adduser $USER lxd` (a supplementary group). This is the common case.
- **Works by staleness:** an unaffected machine, running the same workshop 0.9.4, has a tunnel device created *before* the upgrade that carries **no** `security.uid`/`security.gid` keys, so `forkproxy` keeps its previous (root) privileges and the tunnel works:
```
$ lxc --project workshop.1993010624 profile show dev-ffa5204a-lxd-tunnel
devices:
lxd-tunnel_lxd:
bind: instance
connect: unix:/var/snap/lxd/common/lxd/unix.socket
listen: unix:/var/lib/workshop/run/host-lxd/unix.socket
type: proxy
```
Recreating that workshop under 0.9.4 reproduces the bug.
Contributor guide
Research direction
Start with internal/workshop/lxd/lxd_backend.go at proxyToLxdDevice and internal/interfaces/lxd_device/spec.go at addProxyEntry; inspect how tunnel proxy credentials are derived, then review internal/osutil/user.go. Reproduce with the workshop.yaml configuration and workshop launch/exec commands. Done means the tunneled host LXD socket accepts the first lxc or curl request for supplementary-group users, with regression coverage if the repository provides a suitable test location.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- infrastructure, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 56/100