apache / apache/gravitino

[Improvement]: Use ranger-distro-2.4.0 packages to build ranger and hive image

Open
#7,084 0 comments 0 reactions 1 assignee Claimed by @unknowntpo View on GitHub
improvement
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### What would you like to be improved?

Use packages from `ranger-distro-2.4.0` to build `hive`, `ranger` docker image.

Reference: [ranger-distro-2.4.0 in maven](https://repo.maven.apache.org/maven2/org/apache/ranger/ranger-distro/2.4.0/)

### How should we improve?

Because in my previous PR #6848, ranger packages are built with docker compose, and this will fail because of permission error.

```
#19 [ranger-build] exporting to image
#19 exporting layers
#19 exporting layers 2.2s done
#19 writing image sha256:48a72650eff8944e83665179802cc87b82f6eaeb1198082b6dafcd1c0130d70c done
#19 naming to docker.io/library/ranger-build done
Network rangernw Creating
#19 DONE 2.2s
Network rangernw Created
Container ranger-base Creating
Container ranger-base Created
Container ranger-build Creating
Container ranger-build Created
Attaching to ranger-base, ranger-build

ranger-base exited with code 0
ranger-build | Building release-ranger-2.4.0 branch from https://github.com/apache/ranger.git
ranger-build | Cloning into 'ranger'...
ranger-build | Note: switching to '50ad9c19e62f5aa6b2888b17437aa6fb1e2c0a36'.
ranger-build |
ranger-build | You are in 'detached HEAD' state. You can look around, make experimental
ranger-build | changes and commit them, and you can discard any commits you make in this
ranger-build | state without impacting any branches by switching back to a branch.
ranger-build |
ranger-build | If you want to create a new branch to retain commits you create, you may
ranger-build | do so (now or later) by using -c with the switch command. Example:
ranger-build |
ranger-build | git switch -c
ranger-build |
ranger-build | Or undo this operation with:
ranger-build |
ranger-build | git switch -
ranger-build |
ranger-build | Turn off this advice by setting config variable advice.detachedHead to false
ranger-build |
ranger-build | [ERROR] Could not create local repository at /home/ranger/.m2/repository -> [Help 1]
ranger-build | [ERROR]
ranger-build | [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
ranger-build | [ERROR] Re-run Maven using the -X switch to enable full debug logging.
ranger-build | [ERROR]
ranger-build | [ERROR] For more information about the errors and possible solutions, please read the following articles:
ranger-build | [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/LocalRepositoryNotAccessibleException
ranger-build | mv: cannot stat 'target/version': No such file or directory
ranger-build | mv: cannot stat 'target/ranger-*': No such file or directory
```

And even this problem is solved, it still requires about 40 mins to build package, which is too long.

Luckily, I found that in [`ranger-distro-2.4.0`](https://repo.maven.apache.org/maven2/org/apache/ranger/ranger-distro/2.4.0/) of ranger, it contains all packages we need, including `ranger-admin`, `ranger-hdfs`, `ranger-hive`.
(previously we seek into [ranger-2.4.0](https://repo.maven.apache.org/maven2/org/apache/ranger/ranger/2.4.0/) these packages are not included).

So we can simply use these packages to build image.

And these are to workflow I've run to verify this approach, and it PASSed.
I also use this image to run integration test locally, it also PASSed.

Hive:
- [Github Action Job to build hive image](https://github.com/unknowntpo/gravitino/actions/runs/14703359386)
- [hive docker image built by Github action for testing](https://hub.docker.com/layers/unknowntpo/gravitino-ci/hive-ranger-distro-github-ci/images/sha256-c18f6f3a76630d9857c8f6888671dd1a11e41c26d4e224d019a0c85ce161726e)

Ranger:
- [Github Action Job to build ranger image](https://github.com/unknowntpo/gravitino/actions/runs/14692491562)
- [ranger docker image built by Github action for testing](https://hub.docker.com/layers/unknowntpo/gravitino-ci/ranger-ranger-distro-github-ci/images/sha256-768edf5b5ff7bca9ceb33b2980e9864e51b4bc91a7cc9fc50a4bcb3a53be91fe)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.