apache / apache/hudi

Ensure lock providers are thread safe

Open
#16,943 1 comment 0 reactions 1 assignee Claimed by @nsivabalan View on GitHub
from-jira priority:critical type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

Only InProcessLockProvider is thread safe as of now. The other lock providers are not thread safe. The Jira aims to ensure all the lock providers which support thread safety are made thread safe.
- InProcessLockProvider uses ReentrantReadWriteLock which inherently is thread safe.
- DynamoDBBasedLockProvider uses AmazonDynamoDBLockClient which is thread safe but it stores the lock item as an instance variable which can cause issues with multiple threads.
- Similarly ZookeeperBasedLockProvider uses CuratorFramework which is thread safe but it stores InterProcessMutex as instance variable. The InterProcessMutex is then released during unlock.
- HiveMetastoreBasedLockProvider uses IMetaStoreClient which inherently is not thread safe.
- FileSystemBasedLockProvider is also not thread safe since it stores LockInfo and currentOwnerLockInfo as instance variables.

 

Currently lock provider is closed during unlock and if we are sharing the lock provider amongst multiple threads, we would need to fix the close semantics for lock provider as well. 

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-9254
- Type: Bug
- Fix version(s):
- 1.1.0

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.