awslabs / awslabs/graphstorm

[GSProcessing] Put Entry Script inside the Docker to avoid version mismatch

Open
#1,330 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
452
Forks
76
PR merge metrics
No merged PRs in 30d

Description

Issue we have observed because distributed_executor.py contains lots of code and sometimes includes breaking changes between versions:

* The entry point is uploaded and attached during job launch/pipeline creation, from local graphstorm repo
* If a user cloned the graphstorm repo from main, they get the latest version of the entry point, which in might contain a backwards incompatible change
* When launching a sagemaker job, it will use the latest entry point, on a container that was built using e.g v0.4.2
* As a result, it's looking for library function that was renamed, causing an error.

Ideally the entry point should be very thin, containing argument parsing and one library function call which doesn't change

Any backwards-incompatible changes to the entry point need to marked as breaking (e.g. changing the name of one of its imports)

Contributor guide

Open the contributing guide

Research direction

Start by tracing how distributed_executor.py is uploaded during job launch or pipeline creation and how the SageMaker container uses it. Review the Docker/container version boundary and define the stable library call the entry script should use. Done means the entry point no longer risks mismatching the container version, with breaking entry-point changes explicitly marked.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python
Domain
cloud, distributed-systems, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.