allenai / allenai/olmocr

[bug] Too Many Open Files

Abierto
#429 3 comentarios 0 reacciones 1 asignado Reclamado por @jakep-allenai Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
19.5k
Forks
1.6k
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

### 🐛 Describe the bug

I am getting the too many open files error again, on latest release with 28k files being sent to be converted.

```sh
ping 20s
2026-01-24 20:28:38,065 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-21 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,065 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-20 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,065 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-10 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,065 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-18 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,065 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-27 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,065 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-22 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,067 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-30 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,067 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-28 attempt 0: OSError: [Errno 24] Too many open files. Backoff 2/10, sleeping 20s
2026-01-24 20:28:38,067 - __main__ - WARNING - Connection error on ./pdfs/pdfDownload8/doc6483536.pdf-29 attempt 0: OSError: [Errno 24] Too many open file
```

### Versions

```
argcomplete==3.1.4
attrs==23.2.0
Babel==2.10.3
bcc==0.29.1
blinker==1.7.0
Brlapi==0.8.5
certifi==2023.11.17
chardet==5.2.0
click==8.1.6
cloud-init==25.2
colorama==0.4.6
command-not-found==0.3
configobj==5.0.8
cryptography==41.0.7
cupshelpers==1.0
dbus-python==1.3.2
defer==1.0.6
distro==1.9.0
distro-info==1.7+build1
gpg==1.18.0
httplib2==0.20.4
idna==3.6
Jinja2==3.1.2
jsonpatch==1.32
jsonpointer==2.0
jsonschema==4.10.3
language-selector==0.1
launchpadlib==1.11.0
lazr.restfulclient==0.14.6
lazr.uri==1.0.6
louis==3.29.0
Markdown==3.5.2
markdown-it-py==3.0.0
MarkupSafe==2.1.5
mdurl==0.1.2
netaddr==0.8.0
oauthlib==3.2.2
olefile==0.46
packaging==24.0
pexpect==4.9.0
pillow==10.2.0
pipx==1.4.3
platformdirs==4.2.0
psutil==5.9.8
ptyprocess==0.7.0
pycairo==1.25.1
pycups==2.0.1
Pygments==2.17.2
PyGObject==3.48.2
PyJWT==2.7.0
pyparsing==3.1.1
pyrsistent==0.20.0
pyserial==3.5
python-apt==2.7.7+ubuntu5.1
python-dateutil==2.8.2
python-debian==0.1.49+ubuntu2
pytz==2024.1
pyxdg==0.28
PyYAML==6.0.1
requests==2.31.0
rich==13.7.1
screen-resolution-extra==0.0.0
setuptools==68.1.2
six==1.16.0
ssh-import-id==5.11
systemd-python==235
typing_extensions==4.10.0
ubuntu-drivers-common==0.0.0
ubuntu-pro-client==8001
ufw==0.36.2
unattended-upgrades==0.1
urllib3==2.0.7
userpath==1.9.1
wadllib==1.3.6
wheel==0.42.0
xdg==5
xkit==0.0.0
```
Python 3.12.3, via conda

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.