eclipse-ee4j / eclipse-ee4j/orb-gmbal

Deadlock during ORB shutdown

Open
#85 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
0
Forks
9
Avg merge
1h 27m
Merged PRs (30d)
2

Description

I found the following two threads in a server with stuck JMX calls:

```
java.lang.Thread.State: BLOCKED (on object monitor)
at org.glassfish.gmbal.impl.ManagedObjectManagerImpl.jmxRegistrationDebug(ManagedObjectManagerImpl.java:1225)
- waiting to lock <0x00000000e3cf8858> (a org.glassfish.gmbal.impl.ManagedObjectManagerImpl)
at org.glassfish.gmbal.impl.MBeanImpl.unregister(MBeanImpl.java:315)
- locked <0x00000000e3df4ab8> (a org.glassfish.gmbal.impl.MBeanImpl)
at org.glassfish.gmbal.impl.JMXRegistrationManager.unregister(JMXRegistrationManager.java:201)
- locked <0x00000000e3f2e628> (a java.lang.Object)
at org.glassfish.gmbal.impl.MBeanTree.unregister(MBeanTree.java:383)
- locked <0x00000000e3cf88a0> (a org.glassfish.gmbal.impl.MBeanTree)
at org.glassfish.gmbal.impl.MBeanTree.unregister(MBeanTree.java:378)
- locked <0x00000000e3cf88a0> (a org.glassfish.gmbal.impl.MBeanTree)
at org.glassfish.gmbal.impl.MBeanTree.clear(MBeanTree.java:419)
- locked <0x00000000e3cf88a0> (a org.glassfish.gmbal.impl.MBeanTree)
at org.glassfish.gmbal.impl.ManagedObjectManagerImpl.init(ManagedObjectManagerImpl.java:322)
at org.glassfish.gmbal.impl.ManagedObjectManagerImpl.close(ManagedObjectManagerImpl.java:344)
at com.sun.corba.ee.impl.orb.ORBImpl.destroy(ORBImpl.java:1516)
at ...

java.lang.Thread.State: BLOCKED (on object monitor)
at org.glassfish.gmbal.impl.MBeanTree.getMBeanImpl(MBeanTree.java:413)
- waiting to lock <0x00000000e3cf88a0> (a org.glassfish.gmbal.impl.MBeanTree)
at org.glassfish.gmbal.impl.ManagedObjectManagerImpl.getFacetAccessor(ManagedObjectManagerImpl.java:746)
- locked <0x00000000e3cf8858> (a org.glassfish.gmbal.impl.ManagedObjectManagerImpl)
at org.glassfish.gmbal.impl.TypeConverterImpl$3.toManagedEntity(TypeConverterImpl.java:435)
at org.glassfish.gmbal.impl.TypeConverterImpl$TypeConverterListBase.toManagedEntity(TypeConverterImpl.java:900)
at org.glassfish.gmbal.impl.AttributeDescriptor.get(AttributeDescriptor.java:110)
at org.glassfish.gmbal.impl.TypeConverterImpl$3.toManagedEntity(TypeConverterImpl.java:436)
at org.glassfish.gmbal.impl.AttributeDescriptor.get(AttributeDescriptor.java:110)
at org.glassfish.gmbal.impl.MBeanSkeleton.getAttribute(MBeanSkeleton.java:526)
at org.glassfish.gmbal.impl.MBeanSkeleton.getAttributes(MBeanSkeleton.java:572)
at org.glassfish.gmbal.impl.MBeanImpl.getAttributes(MBeanImpl.java:362)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttributes(java.management@11.0.8/Unknown Source)
at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttributes(java.management@11.0.8/Unknown Source)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(java.management.rmi@11.0.8/Unknown Source)
at ...
```

The two threads are trying to obtains locks on MBeanTree and ManagedObjectManagerImpl in an inconsistent order, leading to a deadlock. This prevents the ORB (and hence the server) from shutting down.

Contributor guide

Open the contributing guide

Research direction

Start by reading ManagedObjectManagerImpl.java, MBeanImpl.java, MBeanTree.java, and JMXRegistrationManager.java at the stack-trace locations, then compare the lock order in shutdown and JMX attribute access. Reproduce the stuck-server scenario with concurrent JMX calls during ORBImpl.destroy; done means ORB shutdown completes without the opposing monitor waits.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.