Discuss Jstack dumps and how we can make interpreting them easier.
- Dominant language
- Java
- Stars
- 14.9k
- Forks
- 3.5k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 88
Description
[Read this](http://blog.takipi.com/supercharged-jstack-how-to-debug-your-servers-at-100mph/)
TL;DR
We do rename the thread. The article suggests running a watchdog thread that would run dumps from within LS and logs the captured output. The dump is triggered by a performance degradation.
We could do this and improve on it by taking one dump when performance is nominal and several in succession when its not. We could log to a different file (if log file arg ti set) by adding a `jstack` suffix to the file name or maybe its better to use the cabin LS logger.
We should also use `LOGSTASH_CORE_VERSION` in the name of the watchdog thread.
Also we should build a knowledge base in google docs of as many LS jstack dumps as we can and put them in versioned folders.
What else can we do?
With the PQ, each job (batch) and event will probably have a unique identifier. Should we at least INFO log the job id as it enters the worker thread? Should each filter/output plugin INFO log each event id as it processes it. <- This maybe useful when the jstack dumps are mingled with LS logging.
Contributor guide
Research direction
Start with the linked jstack article and the issue's proposals for watchdog-triggered dumps, logging, thread naming, and job or event identifiers. No repository files or tests are named; done would require an agreed, scoped design for collecting and interpreting Logstash jstack dumps rather than an open-ended discussion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100