google / google/etils

Exclude TSUBAME /gs/bs /gs/fs from GCS mapping in epath.Path

Open
#734 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
259
Forks
34
Avg merge
57m
Merged PRs (30d)
1

Description

The epath.Path provided by this repository unconditionally treats any path starting with /gs/... as a Google Cloud Storage path (gs://).

However, on the TSUBAME supercomputer provided by the Institute of Science Tokyo (formerly Tokyo Institute of Technology) in Japan, the /gs/bs and /gs/fs paths are used for local storage.
Reference: https://www.t4.cii.isct.ac.jp/docs/handbook.en/storage/

Because of this, the current behavior causes significant problems when running experiments that use this library on the TSUBAME environment, as the local storage paths /gs/bs and /gs/fs are incorrectly interpreted as GCS paths.

I would like to request that /gs/bs and /gs/fs paths be excluded from this automatic conversion and left as-is. If needed, I am also able to submit a PR for this change.

Thank you very much for your consideration.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.