jenkinsci / jenkinsci/support-core-plugin

[JENKINS-24871] Track/detect/analyze contended lock

Open
#715 0 comments 0 reactions 0 assignees View on GitHub
component:support-core-plugin imported-jira-issue priority:major resolution:unresolved
Dominant language
Java
Stars
19
Forks
77
Avg merge
17h 53m
Merged PRs (30d)
3

Description

Often, the extreme slowness in the UI or executors come from lock contention.

Today, I catch them mainly by looking at thread dumps and noticing large number of threads waiting for the same lock. But thread dump is a fairly crude tool for this purpose.


  • If the average duration of one locking is very long, it won't take too many threads to cause a long delay, but they won't be as visible in thread dumps.

  • We need to ask the thread dumps be taken while the problem is exhibiting itself.

This seems like something a tool can help. I don't know what is easy to do, what's possible, and what isn't, but some ideas include:


  • track contended locks across JVMs over time and report them?

  • If doing it across the JVMs is difficult, maybe we can do it for specific objects or specific classes?

  • Or instead of doing this proactively, perhaps the plugin would let us specify an expression to designate an object, then we just monitor that object somehow?

I wonder JMX or Java agent offer anything in this space.

---
Originally reported by kohsuke, imported from: Track/detect/analyze contended lock


  • assignee: schristou
  • status: Open
  • priority: Major
  • component(s): support-core-plugin
  • resolution: Unresolved
  • votes: 0
  • watchers: 1
  • imported: 20251212-090250

Raw content of original issue

Often, the extreme slowness in the UI or executors come from lock contention.

Today, I catch them mainly by looking at thread dumps and noticing large number of threads waiting for the same lock. But thread dump is a fairly crude tool for this purpose.


  • If the average duration of one locking is very long, it won't take too many threads to cause a long delay, but they won't be as visible in thread dumps.

  • We need to ask the thread dumps be taken while the problem is exhibiting itself.

This seems like something a tool can help. I don't know what is easy to do, what's possible, and what isn't, but some ideas include:


  • track contended locks across JVMs over time and report them?

  • If doing it across the JVMs is difficult, maybe we can do it for specific objects or specific classes?

  • Or instead of doing this proactively, perhaps the plugin would let us specify an expression to designate an object, then we just monitor that object somehow?

I wonder JMX or Java agent offer anything in this space.

Contributor guide

Open the contributing guide

Research direction

No source files, tests, or entry points are named. Start by reviewing the support-core-plugin and investigating the JMX or Java agent options raised in the issue; done requires an agreed scope and design for tracking, detecting, and reporting contended locks.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.