MetOffice / MetOffice/lfric_apps
investigate striping of BigData files
- Dominant language
- Fortran
- Stars
- 31
- Forks
- 118
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 8
Description
`lfric_atm` requires a range of static files for initialisation, collectively known as BigData
On Lustre, BigData benefits from striping `lfs (get|set)stripe`
naive striping set all files, whatever the size, to `-c 32 -S 4m -p disk`
a more considered stripin implementation should be investigated, for example:
It is suggested to have the following stripe sizes for file size:
* 1 to 100 MiB - 1 stripe
* 100 MiB to 1GiB - 2 to 4 stripes
* 1 GiB to 100 GiB - 4 to 8 stripes
* It is suggested to use more than 8 stripes for input files if this shows direct benefit in performance tests.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.