apache / apache/curator

[CURATOR-180] Rare occurences of NoNodeException in InterPRocessSemaphoreMurex.acquire

Open
#701 0 comments 0 reactions 0 assignees View on GitHub
bug imported-jira-issue
Dominant language
Java
Stars
3.2k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

On some occasions, acquiring a semaphore will raise a NoNode exception (see stack below). I don't have statistics, but overs 3 weeks with about 20 test jobs/day, I observed it only twice.

If I refer to the code, the parent nodes (here /locks/-licence) should be created if it does not exist:


    private InternalAcquireResult internalAcquire1Lease(ImmutableList.Builder builder, long startMs, boolean hasWait, long waitMs) throws Exception

{
// ... skipping to the problematic lines
try
{
PathAndBytesable<String> createBuilder = client.create().creatingParentsIfNeeded().withProtection().withMode(CreateMode.EPHEMERAL_SEQUENTIAL);
String path = (nodeData != null) ? createBuilder.forPath(ZKPaths.makePath(leasesPath, LEASE_BASE_NAME), nodeData) : createBuilder.forPath(ZKPaths.makePath(leasesPath, LEASE_BASE_NAME));

Here is the observed stack:



org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /locks/-license/leases

at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1590)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:214)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl$3.call(GetChildrenBuilderImpl.java:203)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.pathInForeground(GetChildrenBuilderImpl.java:199)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:191)
at org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:38)
at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.internalAcquire1Lease(InterProcessSemaphoreV2.java:358)
at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.acquire(InterProcessSemaphoreV2.java:281)
at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreV2.acquire(InterProcessSemaphoreV2.java:206)
at org.apache.curator.framework.recipes.locks.InterProcessSemaphoreMutex.acquire(InterProcessSemaphoreMutex.java:46)
at registry.backend.zk.ZooKeeperNodesManager$ZooKeeperRegistryLock.acquire(ZooKeeperNodesManager.java:223)
... 11 more

Note that I can have several processes concurrently trying to grab the lock at the same time. It is also possible that they concurrently try to grab it while the parent path does not yet exist.

---
Originally reported by VGirardReydet, imported from: Rare occurences of NoNodeException in InterPRocessSemaphoreMurex.acquire


  • status: Open
  • priority: Major
  • resolution: Unresolved
  • imported: 2025-01-21

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.