DiamondLightSource / DiamondLightSource/httomo
Multiple tmp dirs created but only one used for file-backed dataset store
- Dominant language
- Python
- Stars
- 10
- Forks
- 5
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 1
Description
This only applies to the case when the data processing has the dataset store backed by an hdf5 file.
In such a case, each process creates a temp dir to hold an hdf5 file: https://github.com/DiamondLightSource/httomo/blob/2acd1b2278a337458a00b63e8528117892eba647/httomo/cli.py#L428-L438
but when the writer actually defines the hdf5 filepath, only rank 0's temp dir is used: https://github.com/DiamondLightSource/httomo/blob/2acd1b2278a337458a00b63e8528117892eba647/httomo/data/dataset_store.py#L174
Meaning, each process creates a temp dir when really only rank 0 needs to create one.
This doesn't have any impact on the functionality, it's simply just a bit confusing when seeing multiple temp dirs created and only one is actually being used.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.