apache / apache/iceberg-python
Cannot connect to a Hive Metastore with Kerberos when the principal's host differs from the URI host
- Ngôn ngữ chính
- Python
- Star
- 1.1k
- Fork
- 581
- Merge trung bình
- 1 ngày 17 giờ
- Pull request đã merge (30 ngày)
- 78
Mô tả
### Apache Iceberg version
0.11.0 (latest release)
### Please describe the bug 🐞
When connecting to a Kerberos-enabled Hive Metastore, authentication fails if the service principal's host does not match the host in the connection URI, and there is no setting to correct it.
I connect to the metastore through three HA hosts:
```python
from pyiceberg.catalog import load_catalog
catalog = load_catalog("hive", **{
"type": "hive",
"uri": "thrift://myhms1:9083,thrift://myhms2:9083,thrift://myhms3:9083",
"hive.kerberos-authentication": "true",
"hive.kerberos-service-name": "myservice",
})
catalog.list_namespaces()
```
The last line fails with:
```
Traceback (most recent call last):
File "test.py", line 24, in
ns = catalog.list_namespaces()
File ".../pyiceberg/catalog/hive.py", line 769, in list_namespaces
with self._client as open_client:
File ".../pyiceberg/catalog/hive.py", line 180, in __enter__
self._transport.open()
File ".../thrift/transport/TTransport.py", line 382, in open
initial_response = self.sasl.process()
File ".../puresasl/client.py", line 148, in process
return self._chosen_mech.process(challenge)
File ".../puresasl/mechanisms.py", line 505, in process
kerberos.authGSSClientStep(self.context, '')
kerberos.GSSError
```
Enabling `KRB5_TRACE` shows that the failure happens when the client asks the KDC for a service ticket:
```
set-error: -1765328243: Did not find credential for myservice/myhms1@EXAMPLE.COM in cache FILE:...
set-error: -1765328377: Error from KDC: LOOKING_UP_SERVER while looking up 'myservice/myhms1@EXAMPLE.COM'
```
My Hive Metastore's service principal is `myservice/hive-host@EXAMPLE.COM`.
PyIceberg, however, requests a ticket using my HMS server hostname (`myhms1`) as the hostname component, which is not a valid service principal, so the request fails.
A Kerberos service principal has the form `Service/Hostname@REALM` (see [3.2 Principal](https://www.kerberos.org/software/tutorial.html#1.3.2) in the Kerberos tutorial), and with SASL/GSSAPI those two components come from the `service` and `host` arguments passed to the client.
PyIceberg always derives that `host` from the connection URI, so it is forced to be whichever host you connect to.
`hive.kerberos-service-name` (added in #2141) makes the `Service` part configurable, but there is no equivalent for `Hostname`.
The relevant line is [`pyiceberg/catalog/hive.py#L167`](https://github.com/apache/iceberg-python/blob/pyiceberg-0.11.x/pyiceberg/catalog/hive.py#L167) (`_HiveClient._init_thrift_transport`):
```python
return TTransport.TSaslClientTransport(socket, host=url_parts.hostname, service=self._kerberos_service_name)
```
There is no way to avoid this from the client side. The `uri` has to contain the hostnames I actually connect to, and the principal's hostname component is not one of them.
Listing multiple URIs does not help either, because the client is built from the first URI and the failure happens later, during authentication.
So I patch that line locally and hardcode the host:
```python
-return TTransport.TSaslClientTransport(socket, host=url_parts.hostname, service=self._kerberos_service_name)
+return TTransport.TSaslClientTransport(socket, host="hive-host", service=self._kerberos_service_name)
```
With that one change everything works well.
PyIceberg requests `myservice/hive-host@EXAMPLE.COM` and `catalog.list_namespaces()` succeeds.
I would like to propose a new configuration property, `hive.kerberos-service-host`. It could be added in the same way as [#2141](https://github.com/apache/iceberg-python/pull/2141).
When the property is not set, the URI host would be used as the default, so the current behavior is preserved for backward compatibility.
I have finished the implementation including unit tests, and verified it end-to-end against the same metastore.
If this is welcome, I would like to submit the PR myself.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Hướng nghiên cứu
Bắt đầu tại pyiceberg/catalog/hive.py#L167 trong _HiveClient._init_thrift_transport và xem xét cách host dịch vụ Kerberos được chọn từ URI. Thêm thuộc tính cấu hình được đề xuất, đồng thời giữ host của URI làm giá trị mặc định, sau đó chạy các unit test được đề cập trong issue và xác minh hành vi kết nối Kerberos.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- python
- Lĩnh vực
- authentication, databases
- Loại issue
- Lỗi
- Độ khó
- 2/5
- Thời gian dự kiến
- 1-3 giờ
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 78/100