apache / apache/iceberg-rust

Upgrade integration test Hive version

Open
#3,175 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
1.4k
Forks
567
Avg merge
2d 2h
Merged PRs (30d)
93

Description

## Context

The HMS integration-test fixture currently uses `apache/hive:3.1.3` with Hadoop 3.1.0 and Tez 0.9.1.

Apache Hive declared the entire 3.x release line end-of-life on 8 October 2024. Hive 3.1.3 is now available only from the Apache historical archive and no longer receives fixes or security updates.

- Hive release status: https://hive.apache.org/general/downloads/
- Hive 3.1.3 archive: https://archive.apache.org/dist/hive/hive-3.1.3/

https://github.com/apache/iceberg-rust/pull/3174 removes the fixture build’s dependency on expired Debian Bullseye package metadata, but it intentionally retains Hive 3.1.3 to avoid changing HMS behavior while unblocking CI.

## Problem

Rebuilding Hive 3.1.3 on a maintained operating-system image would address the EOL base image, but Hive itself and its bundled dependencies would remain unsupported.

The durable solution is to migrate the test fixture and Rust HMS client to a supported Hive 4 release.

This is not currently a drop-in image update. An experiment with `apache/hive:4.0.1` showed:

- The Hive Metastore starts successfully with matching Hadoop 3.3.6 S3 dependencies.
- Basic HMS namespace operations pass.
- Broader catalog coverage fails during table operations with:

```text
Invalid method name: 'get_table'

Contributor guide

Open the contributing guide

Research direction

Locate the HMS integration-test fixture and Rust HMS client, then reproduce the experiment with apache/hive:4.0.1 and matching Hadoop 3.3.6 S3 dependencies. Start with the failing table-operation coverage around the Invalid method name: 'get_table' error. Done means the fixture and client work with a supported Hive 4 release while the broader catalog tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.