DiamondLightSource / DiamondLightSource/httomo

Multiple tmp dirs created but only one used for file-backed dataset store

Open
#699 0 comments 0 reactions 0 assignees View on GitHub
framework minor
Dominant language
Python
Stars
10
Forks
5
Avg merge
5d 17h
Merged PRs (30d)
1

Description

This only applies to the case when the data processing has the dataset store backed by an hdf5 file.

In such a case, each process creates a temp dir to hold an hdf5 file: https://github.com/DiamondLightSource/httomo/blob/2acd1b2278a337458a00b63e8528117892eba647/httomo/cli.py#L428-L438

but when the writer actually defines the hdf5 filepath, only rank 0's temp dir is used: https://github.com/DiamondLightSource/httomo/blob/2acd1b2278a337458a00b63e8528117892eba647/httomo/data/dataset_store.py#L174

Meaning, each process creates a temp dir when really only rank 0 needs to create one.

This doesn't have any impact on the functionality, it's simply just a bit confusing when seeing multiple temp dirs created and only one is actually being used.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.