swagger-api / swagger-api/validator-badge

Too Many Open Files

Open
#138 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
227
Forks
81
Avg merge
2h 3m
Merged PRs (30d)
1

Description

I'm making use of the Docker image for v1.0.5, and am deploying to Kubernetes, and every once in a while the validator spams out a stream of log messages about there being too many open files and then crashes.

Example:

2019-02-14 15:46:42.635:WARN:oejs.ServerConnector:qtp1791741888-16-acceptor-0@18e5ff1-ServerConnector@56ac7bf4{HTTP/1.1}{0.0.0.0:8080}: 
java.io.IOException: Too many open files
        at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
        at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
        at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)
        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Thread.java:748)

We are running a bunch of our own applications that are far more active than the validator is and they never report issues with too many open files, so I don't think it's an issue with the configuration of the Docker Engine.

Looking inside the container, at this particular point in time, I can see that "/validator/webapp/WEB-INF/classes/error.png" is open over 44k times - which doesn't seem right.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by examining the validator deployment and the reported path /validator/webapp/WEB-INF/classes/error.png, then trace the Java server activity shown in the Jetty ServerConnector stack trace. Reproduce the open-file growth if possible and verify that repeated access to error.png no longer exhausts file descriptors or crashes the validator.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java, kubernetes
Domain
backend, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.