Experiencing Exception on Hive/SASL/auth-conf
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
When accessive Hive through SQLAlchemy or the Jupyter Hive Kernel, using Kerberos and the auth-conf setting, we are getting the following exception.
It occurs on a simple select statement listing 300 rows:
```
Traceback (most recent call last):
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1211, in fetchall
l = self.process_rows(self._fetchall_impl())
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1161, in _fetchall_impl
return self.cursor.fetchall()
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/common.py", line 136, in fetchall
return list(iter(self.fetchone, None))
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/common.py", line 105, in fetchone
self._fetch_while(lambda: not self._data and self._state != self._STATE_FINISHED)
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/common.py", line 45, in _fetch_while
self._fetch_more()
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/pyhive/hive.py", line 386, in _fetch_more
response = self._connection.client.FetchResults(req)
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 714, in FetchResults
return self.recv_FetchResults()
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/TCLIService/TCLIService.py", line 726, in recv_FetchResults
(fname, mtype, rseqid) = iprot.readMessageBegin()
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 134, in readMessageBegin
sz = self.readI32()
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift/protocol/TBinaryProtocol.py", line 217, in readI32
buff = self.trans.readAll(4)
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift/transport/TTransport.py", line 60, in readAll
chunk = self.read(sz - have)
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 173, in read
self._read_frame()
File "/var/log/hadoop/anaconda3/lib/python3.6/site-packages/thrift_sasl/__init__.py", line 187, in _read_frame
message=self.sasl.getError())
thrift.transport.TTransport.TTransportException: b'Error in sasl_decode (-1) SASL(-1): generic failure: Unable to find a callback: 32775'
```
Turns out this is generated by the Cyrus-SASL native library.
I am able to fix the issue by specifying a larger value for 'maxbufsize' in the sasl_factory in hive.py. The default is 65535 bytes or so.
```
def sasl_factory():
sasl_client = sasl.Client()
sasl_client.setAttr('host', host)
if sasl_auth == 'GSSAPI':
sasl_client.setAttr('service', kerberos_service_name)
#+B. Gagnon 2021-07-12: patch to support auth-conf
sasl_client.setAttr('maxbufsize', 2*1024*1024)
#-B. Gagnon 2021-07-12: patch to support auth-conf
elif sasl_auth == 'PLAIN':
sasl_client.setAttr('username', username)
sasl_client.setAttr('password', password)
else:
raise AssertionError
sasl_client.init()
return sasl_client
self._transport = thrift_sasl.TSaslClientTransport(sasl_factory, sasl_auth, socket)
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.