apache / apache/iceberg-python

Support DIGEST-MD5 / delegation token authentication for HMS

Open
#3,145 1 comment 0 reactions 0 assignees View on GitHub
stale
Dominant language
Python
Stars
1.1k
Forks
581
Avg merge
1d 17h
Merged PRs (30d)
77

Description

### Feature Request / Improvement

#### Summary
PyIceberg's HiveCatalog supports Kerberos (GSSAPI) authentication via `hive.kerberos-authentication`, but does not support DIGEST-MD5 SASL authentication with Hadoop delegation tokens. In many production Hadoop environments, pods/containers authenticate to HMS using delegation tokens (read from `$HADOOP_TOKEN_FILE_LOCATION`) rather than Kerberos keytabs. This means PyIceberg's Hive catalog cannot be used in these environments without building a custom client.

#### Proposed Enhancement
Extend `_HiveClient` to support DIGEST-MD5 delegation token auth:
1. Add a new config property (e.g. `hive.metastore.authentication=DIGEST-MD5`)
2. When DIGEST-MD5 is configured, read credentials from `$HADOOP_TOKEN_FILE_LOCATION` (Hadoop Writable credentials format)
3. Use `TSaslClientTransport` with mechanism=DIGEST-MD5 and the extracted token identifier/password

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with _HiveClient and its existing hive.kerberos-authentication path, then inspect how TSaslClientTransport is configured. The work is done when DIGEST-MD5 reads the Hadoop Writable credentials from $HADOOP_TOKEN_FILE_LOCATION and uses the token identifier and password for HMS authentication.

Written by the indexing model from the issue text.

Assessment

Tech stack
hadoop, python
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.