EMFILE: too many open files
- Dominant language
- JavaScript
- Stars
- 20.8k
- Forks
- 2.3k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 156
Description
Describe the bug
Hi there, first things first sorry if my english isn't proper and my questions isn't clear or I miss a requirement for raising up an issue. I just started using Svelte with Svelte Kit for an app on production, but recently we started to get some crashes on the server.
All the times the crashes happens, the log just said as how I've put the log below, just maybe with a different source of files, but mostly from the same immutable directory. This app is served inside a container that's managed by CapRover.
I've tried increasing the number of instances of container for the app and increasing the container's open file limit, but this problem still occurs.
There is too much files that are being opened by node and it keeps increasing each time until it hits the limits, so I personally don't think that keep increasing the ulimit will be the proper solution here, for example:
1 /usr/local/bin/node 4107 /application/build/client/_app/immutable/assets/0.78852c93.css
1 /usr/local/bin/node 4108 /application/build/client/_app/immutable/nodes/0.0b071741.js
1 /usr/local/bin/node 4109 /application/build/client/_app/immutable/assets/2.881459ec.css
1 /usr/local/bin/node 4110 /application/build/client/_app/immutable/chunks/stores.b3a6fc4b.js
1 /usr/local/bin/node 4111 /application/build/client/_app/immutable/chunks/ProgressBar.svelte_svelte_type_style_lang.d0fe8878.js
1 /usr/local/bin/node 4112 /application/build/client/_app/immutable/chunks/index.3b58e18e.js
1 /usr/local/bin/node 4113 /application/build/client/_app/immutable/entry/start.a85984a3.js
1 /usr/local/bin/node 4114 /application/build/client/_app/immutable/entry/app.5c119c71.js
1 /usr/local/bin/node 4116 /application/build/client/_app/immutable/nodes/2.45c747de.js
1 /usr/local/bin/node 4117 /application/build/client/_app/immutable/chunks/stores.b3a6fc4b.js
1 /usr/local/bin/node 4118 /application/build/client/_app/immutable/chunks/index.3b58e18e.js
1 /usr/local/bin/node 4119 /application/build/client/_app/immutable/nodes/2.45c747de.js
1 /usr/local/bin/node 4120 /application/build/client/_app/immutable/nodes/0.0b071741.js
1 /usr/local/bin/node 4121 /application/build/client/_app/immutable/assets/2.881459ec.css
Is something that I've dong wrong here? Maybe my Dockerfile, or how I handled an import? etc? anyone else having a similar issue?
By the way, here's the Dockerfile that I'm using:
FROM node:alpine AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable
COPY . /application
WORKDIR /application
FROM base AS builder
WORKDIR /application
COPY package.json .
COPY pnpm-lock.yaml .
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prefer-frozen-lockfile
COPY . .
ARG ENV A
ARG ENV B
ARG ENV C
ARG NODE_ENV
ARG PUBLIC_ENV_A
ARG PUBLIC_ENV_B
ARG PUBLIC_ENV_C
ARG SENTRY_AUTH_TOKEN
RUN pnpm run build
FROM base as dependencies
RUN --mount=type=cache,id=pnpm,target=/pnpm/store pnpm install --prefer-frozen-lockfile -P
FROM base
WORKDIR /application
COPY --from=builder /application/build build/
COPY --from=dependencies /application/node_modules node_modules/
COPY package.json .
ARG NODE_ENV
EXPOSE 3000
CMD [ "node", "-r", "dotenv/config", "build" ]
Thanks a lot beforehand, and sorry for the inconveniences if my questions are incomplete or just blatantly wrong here.
Reproduction
I can't really reproduce it since usually this problem occurs when we're having a bit a peak traffic (around 100-ish visitor)
Logs
2024-01-12T03:43:04.213873821Z [Error: EMFILE: too many open files, open '/application/build/client/_app/immutable/chunks/ProgressBar.svelte_svelte_type_style_lang.d0fe8878.js'] {
2024-01-12T03:43:04.213911498Z errno: -24,
2024-01-12T03:43:04.213914657Z code: 'EMFILE',
2024-01-12T03:43:04.213916946Z syscall: 'open',
2024-01-12T03:43:04.213929631Z path: '/application/build/client/_app/immutable/chunks/ProgressBar.svelte_svelte_type_style_lang.d0fe8878.js'
2024-01-12T03:43:04.213932569Z }
2024-01-12T03:45:03.273075549Z [Error: EMFILE: too many open files, open '/application/build/client/_app/immutable/entry/app.5c119c71.js'] {
2024-01-12T03:45:03.273131966Z errno: -24,
2024-01-12T03:45:03.273135314Z code: 'EMFILE',
2024-01-12T03:45:03.273137644Z syscall: 'open',
2024-01-12T03:45:03.273139860Z path: '/application/build/client/_app/immutable/entry/app.5c119c71.js'
2024-01-12T03:45:03.273142231Z }
System Info
Alpine Linux: 3.19.0
Architecture: arm64
Node: 21.5.0
Containerized inside a node:alpine image, using latest version of Node and running on an ARM Architecture.
I'm also using pnpm to build the app, and requiring dotenv as the documentation suggested when running the app.
Severity
annoyance
Additional Information
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Dockerfile and the runtime command node -r dotenv/config build, then inspect how the generated build/client/_app/immutable files are served. Reproduce the file-descriptor growth under concurrent traffic if possible and compare it with the linked pull request. Done means the cause of the repeated open files is identified and the issue has a verified fix or documented configuration requirement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, javascript, node.js
- Domain
- backend, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100