DiamondLightSource / DiamondLightSource/httomo
Sort out inconsistencies and oddities with the loader's YAML template and how a loader is referenced in a pipeline file
- Dominant language
- Python
- Stars
- 10
- Forks
- 5
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 1
Description
## Current situation with the loader and its YAML template
The history of the loader YAML template is a bit weird, so it may be worth summarising here:
- we had the `standard_tomo()` _function_ in the `httomo.data.hdf.loaders` module, and the YAML template for the loader was generated from that function
- when the block loader was introduced, namely, the `StandardTomoLoader` _class_ in the `httomo.loaders.standard_tomo_loader` module, we didn't know how to tackle generating a YAML template from the class, so we pushed the problem to the side and simply:
- kept `httomo.data.hdf.loaders` in the YAML template and pipeline
- in the refactoring of httomo where the block loader was introduced, we basically treat the very first method in the pipeline as a special case, and thus we do not adhere to the `module_path` and `method` config for the loader in the YAML pipeline
The situation we have now is that the YAML config for a loader is inconsistent with where the loader code is actually located (ie, it's in `httomo.loaders.standard_tomo_loader` rather than `httomo.data.hdf.loaders`, despite the config implying the latter).
## Stuff to address (they're somewhat connected, rather than being totally independent problems)
### Loader YAML template generation
Due to the block loader being a class and not a function, enabling the YAML template generator script to work with loaders which are classes and not functions will probably require some tweaks to it. For example, maybe by checking if `httomo.loaders` is part of the module path, loader config in the pipeline can be caught and then treated differently to methods (which are just functions)?
### Referencing a loader in a pipeline file
The old loader (ie, the loader which didn't load individual blocks) in `httomo.data.hdf.loaders` is not used anymore, yet pipelines all reference `httomo.data.hdf.loaders`. To be consistent with how methods in a YAML pipeline file are referenced via their:
- module path
- method function name
it would make sense to have the way to reference an httomo loader in a pipeline to be something like:
```yaml
- method: StandardTomoLoader
module_path: httomo.loaders.standard_tomo_loader
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.