abiosoft / abiosoft/colima

Attempting to open >256 files results in `Operation not permitted`

Aperta
#911 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Go
Stelle
30.8k
Fork
613
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

### Description

I encountered this while doing a `cargo build`, while `cargo` was attempting to build `openssl`. OpenSSL's build includes a `ar` command that archives quite a lot of `.o` files into one archive. This isn't particularly unusual.

However, that process fails:
```
make: Warning: File 'Makefile' has modification time 1.5 s in the future
make: warning: Clock skew detected. Your build may be incomplete.
ar: crypto/asn1/libcrypto-lib-d2i_pr.o: Operation not permitted
make[1]: *** [Makefile:4160: libcrypto.a] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:2134: build_libs] Error 2
```

If we attempt to replicate the failing `ar` command under an `strace`:

```
newfstatat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_key.o", {st_mode=S_IFREG|0644, st_size=56776, ...}, 0) = 0
openat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_key.o", O_RDONLY) = 251
fcntl(251, F_GETFD) = 0
fcntl(251, F_SETFD, FD_CLOEXEC) = 0
newfstatat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_lib.o", {st_mode=S_IFREG|0644, st_size=44552, ...}, 0) = 0
brk(0x5568116c3000) = 0x5568116c3000
openat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_lib.o", O_RDONLY) = 252
fcntl(252, F_GETFD) = 0
fcntl(252, F_SETFD, FD_CLOEXEC) = 0
newfstatat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_meth.o", {st_mode=S_IFREG|0644, st_size=25328, ...}, 0) = 0
openat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_meth.o", O_RDONLY) = 253
fcntl(253, F_GETFD) = 0
fcntl(253, F_SETFD, FD_CLOEXEC) = 0
newfstatat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_pmeth.o", {st_mode=S_IFREG|0644, st_size=73688, ...}, 0) = 0
openat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_pmeth.o", O_RDONLY) = 254
fcntl(254, F_GETFD) = 0
fcntl(254, F_SETFD, FD_CLOEXEC) = 0
newfstatat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_prn.o", {st_mode=S_IFREG|0644, st_size=9936, ...}, 0) = 0
openat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_prn.o", O_RDONLY) = 255
fcntl(255, F_GETFD) = 0
fcntl(255, F_SETFD, FD_CLOEXEC) = 0
newfstatat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_rfc5114.o", {st_mode=S_IFREG|0644, st_size=12280, ...}, 0) = 0
openat(AT_FDCWD, "crypto/dh/libcrypto-lib-dh_rfc5114.o", O_RDONLY) = -1 EPERM (Operation not permitted)
write(2, "ar: crypto/dh/libcrypto-lib-dh_r"..., 66ar: crypto/dh/libcrypto-lib-dh_rfc5114.o: Operation not permitted
) = 66
```
We see that the prior successes end with the successful opening of file descriptor `255`, and the next `openat` fails, which is a suspect point to fail at. `ulimit` indicates that's not the problem, and all these files have the same owner/perms, so a real permission error is also not the problem.

We can replicate this quite simply by just attempting to `ar` 300 empty files together, and thus remove OpenSSL from the repro test:

1. Start a container; mount a host directory & cd into it. (This command does all that.) I'm trying to actually get the output of the build once it finishes, of course… 😉
```
docker run --rm -ti -w /src --mount type=bind,src=$(pwd),dst=/src rust:slim-bookworm
```

2. Attempt to do the `ar`:
```
# mkdir foo && cd foo && for i in `seq 1 300`; do touch $i; done; ar qc test.a *
ar: 56: Operation not permitted
```
N.b. that "56" ends up being the 256th file. The glob seems to pass the files in lexicographic order, so we open 1, 10, 100, 101, etc. first. `strace`:
```
# Grab strace:
apt-get update && apt-get install --yes --no-install-recommends strace
# strace:
strace ar qc test.a *
[snip]
openat(AT_FDCWD, "55", O_RDONLY) = 255
fcntl(255, F_GETFD) = 0
fcntl(255, F_SETFD, FD_CLOEXEC) = 0
newfstatat(AT_FDCWD, "56", {st_mode=S_IFREG|0644, st_size=0, ...}, 0) = 0
openat(AT_FDCWD, "56", O_RDONLY) = -1 EPERM (Operation not permitted)
```

### Version

```
» colima version && limactl --version && qemu-img --version
colima version 0.5.6
git commit: ceef812c32ab74a49df9f270e048e5dced85f932
limactl version 0.18.0
qemu-img version 8.1.2
Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers
```

### Operating System

- [X] macOS Intel <= 13 (Ventura)
- [ ] macOS Intel >= 14 (Sonoma)
- [ ] Apple Silicon <= 13 (Ventura)
- [ ] Apple Silicon >= 14 (Sonoma)
- [ ] Linux

### Output of `colima status`

(`colima` started failing to start for me while filing this bug report. So, now I'm going to attempt to figure out what bug *that* is…)

Edit: Okay, working again, still bugged:
```
» colima status
INFO[0000] colima is running using QEMU
INFO[0000] arch: x86_64
INFO[0000] runtime: docker
INFO[0000] mountType: sshfs
INFO[0000] socket: unix:///Users/roy/.colima/default/docker.sock
```

### Reproduction Steps

Open >256 files in a single process inside the container.

### Expected behaviour

It works.

### Additional context

_No response_

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

L’issue riguarda un limite dei descrittori di file durante l’apertura di più di 256 file all’interno di un container. Esamina il codice di colima che configura l’ambiente del container, possibilmente nella configurazione della VM o nella gestione dei mount. L’output di strace mostra il fallimento a FD 255. Indaga su come colima configura i limiti delle risorse o i mount del file system. Verifica se esiste un limite noto nel livello sottostante di lima o QEMU. Riproduci il bug con i passaggi forniti per confermare l’issue.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
docker, go, linux
Ambito
cli, operating-systems
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Specificata chiaramente
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.