google-deepmind / google-deepmind/xmanager

Mounting GS storage bucket inside containers

Open
#36 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
921
Forks
69
Avg merge
1d 1h
Merged PRs (30d)
1

Description

I am trying to launch experiments locally with mounted data stored on GCS. Eventually I would like to run the experiments remotely on Kubernetes/VertexAI so I'd like the container to mount the data itself instead of mounting the data on the host machine manually.
I am using prebuilt containers by passing in the image path to the Container executable type. My entrypoint script mounts the GS data buckets with gcsfuse and runs the python training script. However, when running my containers through the xmanager launch script the gcsfuse operation exits with code 1. Running the containers manually with the --privileged option seemed to resolve the issue. However, there currently doesn't exist a xmanager DockerOption to specify `--privileged` mode.
Is it possible to add such an option? Or is there a different recommended way to mount the data?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.