Auto-mount/unmount storages
- Dominant language
- Java
- Stars
- 163
- Forks
- 65
- Avg merge
- 11h 46m
- Merged PRs (30d)
- 29
Description
**Background**
Currently, newly created storages that become available to a user do not automatically mount in already launched user runs. Similarly, storages that were mounted at the start of a run are not automatically unmounted if they are later deleted during the run.
**Approach**
Implement automatic mounting and unmounting of data storages during a user run:
- **Auto-mount**: New storages that become available to the user after the run has started should be automatically detected and mounted. Storages with changed mount options (permissions) during the run should be re-mount with updated options.
- **Auto-unmount**: Storages that were mounted at the start of the run but are later deleted should be automatically unmounted.
**Technical proposal**
1. Modify the [mount_storage.py](https://github.com/epam/cloud-pipeline/blob/develop/workflows/pipe-common/scripts/mount_storage.py) script to run as a daemon that continuously monitor available storages for the owner run user.
Periodically (based on `CP_CAP_MOUNT_REFRESH_INTERVAL`, default: 5 minutes):
- Fetch the list of storages available to the user.
- Compare with the previously known set.
- Mount newly available storages.
- Unmount storages that have been removed or are no longer accessible.
- Remount storages if necessary when mount option changes are detected:
- permission changes
- mountDisabled status
- region settings changes
2. Combine the [watch_mount_shares.py](https://github.com/epam/cloud-pipeline/blob/develop/workflows/pipe-common/scripts/watch_mount_shares.py) script logic regarding to NFS status monitoring with [mount_storage.py](https://github.com/epam/cloud-pipeline/blob/develop/workflows/pipe-common/scripts/mount_storage.py) script to prevent duplication of code and API calls.
3. Add feature flags to control which parts of the storage monitor are active (default: enable all):
- auto mount/umount
- auto permissions
- auto nfs status
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with workflows/pipe-common/scripts/mount_storage.py and workflows/pipe-common/scripts/watch_mount_shares.py, then trace how these scripts run during a user run. Compare the existing storage and NFS monitoring behavior with the proposed refresh interval and feature flags. Done means one coordinated monitor handles auto-mount, auto-unmount, permission changes, mountDisabled and region changes, plus NFS status monitoring.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100