aws-samples / aws-samples/aws-glue-samples

Request to Host Glue Spark UI Images on DockerHub

Open
#137 0 comments 3 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
834
Avg merge
2d 17h
Merged PRs (30d)
1

Description

Currently I need to write a bunch of code to automatically download and build this docker image because I can't pull it directly from DockerHub.

```shell
echo "glue/sparkui image not found, building image..."
TEMPD=$(mktemp -d -t "glue-sparkui-XXXX")
if [ ! -e "$TEMPD" ]; then
echo >&2 "Failed to create temp directory"
exit 1
fi
cd "$TEMPD"
git clone https://github.com/aws-samples/aws-glue-samples.git
(cd aws-glue-samples/utilities/Spark_UI/glue-1_0-2_0 && docker build -t glue/sparkui:2.0 .)
(cd aws-glue-samples/utilities/Spark_UI/glue-3_0 && docker build -t glue/sparkui:3.0 .)
(cd aws-glue-samples/utilities/Spark_UI/glue-4_0 && docker build -t glue/sparkui:4.0 .)
cd -
rm -rf "$TEMPD"
```

Can these docker images be published directly to DockerHub so I can simply `docker pull` them and start using immediately?

I see that the `aws-glue-libs` are already hosted there: https://hub.docker.com/r/amazon/aws-glue-libs

Would be great to host the Spark UI as well!

Contributor guide

Open the contributing guide

Research direction

Start with the Docker build directories utilities/Spark_UI/glue-1_0-2_0, glue-3_0, and glue-4_0, and review the existing build commands in the issue. Determine how these images could be published to DockerHub alongside the existing amazon/aws-glue-libs images. Done means users can pull the Spark UI images directly instead of cloning the repository and building them.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker
Domain
devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.