tailscale / tailscale/tailscale
Tailscale SSH does not work on Google Cloud Run, `setgroups` fails
- Dominant language
- Go
- Stars
- 36.5k
- Forks
- 3.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 123
Description
### What is the issue?
I tried to open ssh connection from my own computer (Ubuntu 22.04 LTS) to a container running Tailscale on Google Cloud Run service but connection failed with "operation not permitted" error message.
```
masa@masa-x1$ ssh www@100.127.204.106
The authenticity of host '100.127.204.106 (100.127.204.106)' can't be established.
ED25519 key fingerprint is SHA256:nXser+W1F4BwyF12llmr2OsTh78jYJa8zZDV3WP9W4M.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '100.127.204.106' (ED25519) to the list of known hosts.
operation not permitted
Connection to 100.127.204.106 closed.
```
From Google Cloud Run logs I can see that the following command exited with code 1:
```
starting pty command: [/usr/sbin/tailscaled be-child ssh --uid=1000 --gid=1000 --groups=1000 --local-user=www --remote-user=mattirantakomi@github --remote-ip=100.111.215.88 --has-tty=true --tty-name=pts/0 --shell --login-cmd=/usr/bin/login --cmd=/bin/bash -- -l]
```
I tried to debug with strace and found out that the failing syscall is setgroups.
```
...
geteuid() = 1000
geteuid() = 1000
geteuid() = 1000
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=185, si_uid=1000} ---
rt_sigreturn({mask=[]}) = 1000
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=185, si_uid=1000} ---
rt_sigreturn({mask=[]}) = 1000
getegid() = 1000
futex(0xc000060548, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0x15fbd08, FUTEX_WAIT_PRIVATE, 0, NULL) = 0
setgroups(1, [1000]) = -1 EPERM (Operation not permitted)
futex(0x15fc0c0, FUTEX_WAKE_PRIVATE, 1) = 1
futex(0xc000060548, FUTEX_WAKE_PRIVATE, 1) = 1
geteuid() = 1000
getgroups(0, NULL) = 0
write(2, "operation not permitted\n", 24operation not permitted
) = 24
exit_group(1) = ?
+++ exited with 1 +++
```
As far as I know this bug should have been already fixed on https://github.com/tailscale/tailscale/pull/6904 but I'm not sure about it.
### Steps to reproduce
- Get tailscale auth key from Tailscale Admin Web UI -> Settings -> Keys -> Auth keys -> Generate auth key (https://login.tailscale.com/admin/settings/keys). Use generated key as Cloud Run environment variable `TAILSCALE_AUTHKEY`.
- Run test container (https://github.com/mattirantakomi/cloudrun-nginx-test) on Google Cloud Run. Nginx is included and responding on port 8080 just to make Cloud Run health check happy.
```
git clone git@github.com:mattirantakomi/cloudrun-nginx-test.git
gcloud run deploy cloudrun-nginx-test \
--project \
--region europe-north1 \
--service-account \
--set-env-vars TAILSCALE_AUTHKEY="authkey-1234" \
--execution-environment gen2 \
--min-instances=1 --max-instances=1 \
--source .
```
I tried both gen1 and gen2 execution environments, neither doesn't work.
- Try to SSH to that container `ssh www@100.11.22.33`.
### Are there any recent changes that introduced the issue?
-
### OS
Linux
### OS version
Ubuntu 22.04 LTS
### Tailscale version
1.42.0
### Other software
_No response_
### Bug report
BUG-0937c81c9c29b37dc74221ea0bec337231bfcf36999b0a0ff752edde9aa2d8f2-20230621061253Z-1f619bfeaef05cc7
Contributor guide
Assessment
This issue has not been assessed yet.