[Question] Best practice for distributing training code to workers

From the examples provided, it seems that the training script that each MPI worker will execute are bundled in the container images. Is there a better recommended approach for distributing the training code? Ideally looking for something that will allow images to be more static and only contain the required dependencies, while training code can be distributed as part of setting up the job.

Feel free to point me to docs or more relevant material that covers this topic. I just could not find anything myself.

Start by reviewing the repository's provided examples and the job setup material for how MPI workers receive training scripts. Compare the current container-bundling approach with any documented job-time distribution options; done would be a clear recommendation or an explicit statement that no supported alternative exists.

Written by the indexing model from the issue text.