Ensure lock providers are thread safe
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Only InProcessLockProvider is thread safe as of now. The other lock providers are not thread safe. The Jira aims to ensure all the lock providers which support thread safety are made thread safe.
- InProcessLockProvider uses ReentrantReadWriteLock which inherently is thread safe.
- DynamoDBBasedLockProvider uses AmazonDynamoDBLockClient which is thread safe but it stores the lock item as an instance variable which can cause issues with multiple threads.
- Similarly ZookeeperBasedLockProvider uses CuratorFramework which is thread safe but it stores InterProcessMutex as instance variable. The InterProcessMutex is then released during unlock.
- HiveMetastoreBasedLockProvider uses IMetaStoreClient which inherently is not thread safe.
- FileSystemBasedLockProvider is also not thread safe since it stores LockInfo and currentOwnerLockInfo as instance variables.
Currently lock provider is closed during unlock and if we are sharing the lock provider amongst multiple threads, we would need to fix the close semantics for lock provider as well.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-9254
- Type: Bug
- Fix version(s):
- 1.1.0
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.