Improve QEMU-based container host (Docker Desktop) compatibility
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 15h 13m
- Merged PRs (30d)
- 368
Description
When using Docker Desktop for Mac or other container host based on QEMU VMs, you may see errors like:
```Java
invalid mount config for type "bind": bind source path does not exist: /tmp/backend.ai/ipc/container
```
One known workaround so far is to replace `/tmp` with a physically available host directory (such as `$HOME/somewhere`), but recently ([since 4.15 release on Dec 1st 2022](https://docs.docker.com/desktop/release-notes/#4150)), Docker Desktop began to use **virtiofs** as their default filesystem driver which is not compatible with UNIX socket bind-mounts between macOS and the QEMU VM.
We need to upgrade our socket-relay container's `socat` version to 1.7.4 or higher (currently it's 1.7.3) and let it use `VSOCK-LISTEN` instead of `UNIX-LISTEN` with QEMU-based container hosts.
Also we need to patch libzmq and pyzmq to support VSOCK: zeromq/libzmq#4491
Reported-by: @inureyes
ref)
- https://stefano-garzarella.github.io/posts/2021-01-22-socat-vsock/
- https://man7.org/linux/man-pages/man7/vsock.7.html
### Tasks
- [x] #986\* This will be reverted once we have the VSOCK-based socket-relay.
- [ ] Enable vsock-based communication between compute session containers and the socket-relay container (both inside the VM)\* [ ] zeromq/libzmq#4491
- [x] Check if how we can assign and read the CID of containers (not VMs!)\* It turned out that containers share the same CID as long as they run inside a single VM. Nonetheless, we could still utilize VSOCK to implement IPC between containers without fully exposing the host network namespace.
- [x] #988
- [ ] Change the socket-relay container's main command to use `TCP-CONNECT=host.docker.internal` with a separate network namespace instead of `TCP-CONNECT=localhost` with the host-networking mode (not supported in Docker Desktop) when the agent is running with Docker Desktop for Mac/Windows
- [ ] Use metadata server instead of socket proxy\* To incrementally deploy of the metadata server as it requires custom routing, let's make this as an agent's local configuration and other codes relying on this feature to also recognize the option.
- We need to confirm that this migration should have little impact to the performance of other codes relying on this feature.
JIRA Issue: BA-225
Contributor guide
Assessment
This issue has not been assessed yet.