apache / apache/hudi

Make docker not a requirement for building metaserver

Open
#15,757 0 comments 0 reactions 0 assignees View on GitHub
area:dev-experience from-jira priority:high type:improvement
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 4h
Merged PRs (30d)
112

Description

currently to build hudi-platform-service profile, we need docker or a local native thrift binary. see logic related to hudi-platform-service/hudi-metaserver/src/main/thrift/bin/thrift_binary.sh

Docker should not be a requirement for build a java project. Some other issue: the docker generated source files may lack of permission for current user to delete, which could result in mvn clean to fail. encountered this issue with GH actions CI. i have to change the ownership for generated folder with sudo, but this leaves a loophole in the dev build flow - people may run into permission issue when sudo was involved. See this PR https://github.com/apache/hudi/pull/7858/

The goal is to remove the docker requirement for build hudi project, and make thrift generation smooth across different OSes

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-5708
- Type: Improvement

---

## Comments

07/Feb/23 15:02;minihippo;Thanks Raymond for reporting and processing this issue. I have a proposal about how to load thrift to compile the thrift files developer-friendly.

Proposal 1: script for compile and then install  thrift.tar.gz

Proposal 2: provide thrift binary for fixed version OS i.e. centos7, unbantu 18.04, mac OS

For p1, thrift with a specific version will be installed permanently on a developer PC after run once, and the install process cloud have some unexpected problem like the lackness of dependencies.

For p2, it has limitations in usage. I'm not sure it can work, I need to verify a case.

Or does anyone else have a better suggestion about this?

 ;;;

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the hudi-platform-service profile and hudi-platform-service/hudi-metaserver/src/main/thrift/bin/thrift_binary.sh to trace how Thrift sources are generated. Check the linked PR for prior context, then verify that the project builds without Docker or a local native Thrift binary and that generated files can be removed by the current user across supported operating systems.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.