Create emphemeral volumes for inputdir
- Dominant language
- Python
- Stars
- 11
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
Currently, _pfcon_ copies the data of an `inputdir` from "`STOREBASE`", the persistent volume containing _CUBE_'s files, to a temporary `'key-' + job_id` directory. This temporary directory exists in the same volume as `STOREBASE`.
What if _pfcon_ creates an ephemeral volume for the `'key-' + job_id` directory instead of reusing the `STOREBASE`'s volume? This would enable some optimizations in multi-node Kubernetes:
- Ephemeral volumes for jobs can use a different [StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/) from _CUBE_, optimizing performance. Heck, it could even use a memory-backed tmpfs or [RAM drive](https://en.wikipedia.org/wiki/RAM_drive).
- Some clusters lack the ability to provide ReadWriteMany volumes. If the system could copy data out from _CUBE_'s storage to ephemeral volumes, it would be possible to use ReadWriteOnce volumes and benefit from a mulit-node cluster.
- The "delete job" could be an atomic volume delete operation, instead of a `rm -rf`
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.