acquireReadWriteLocks takes more time while running Hive query by PyHive
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I am trying to run my query through PyHive, It takes more time for the execution and I have observed that one of the **acquireReadWriteLocks** takes more time in the execution plan. If that is case Could you please help me how to resolve it.
acquireReadWriteLocks= 3m 37s 10ms (in Total Query execution Duration is 2m 42s 924ms)
acquireReadWriteLocks= 3m 43s 47ms (in Total Query execution Duration is 3m 59s 587ms)
The same Hive Query is getting completed in less than 10 seconds if I try through Hive Beeline
PyHive script to connect Hive.
-----------------------------------
from pyhive import hive
import pandas as pd
hive_conn = hive.connect('ip-address',username='USER-NAME', password='PASSWORD',auth='XXXX')
query="select * from HIVE_DB.TABLE_NAME where date >= '2011-01-01' and date <= '2011-01-02' "
df = pd.read_sql_query(query,con=hive_conn)
hive_conn.close()
Note:
Table is .orc file-format, partitioned by date column
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.