Reduce inodes count
- Dominant language
- Python
- Stars
- 9
- Forks
- 9
- Avg merge
- 12d 12h
- Merged PRs (30d)
- 2
Description
In our CI pipeline we are constantly running out of inodes (if a CI run fails, there is no cleanup). I expect that Fab will put additional pressure on the number of inodes, since for each (Fortran) input file, it will create:
1. a preprocessed file (even if the input file is .f90, it will create a copy so that the whole build is done from `build_dir` only)
2. an `.an` file
3. two `.mod` files (one with hash, one without)
4. a `.o` file
Though it also does save by only compiling the absolute minimum number of files. A rough analysis of one of my gungho builds:
```
930 source files
4676 build files
1156 *90 files (note that PSyclone will create two Fortran files for one X90 input file)
1675 *mod files (that's two per *90 file - hashed and non-hased version. Not all *90 files will probably be used in gungho, so that's why we have only around 837.5 files ... and no idea about why this is an odd number, best guess would be left-over mod files, since I rebuild in the same directory, and my fab development version sometimes crashes before cleanup??)
820 *o files
1023 .an files (analysis files)
```
So, of the 5606 overall files, the .hashed mod file and the .an file might not need to be written, saving around 1840 inodes, or around 33% (Fab uses internally cached information to get the analysis information from the analysis step to the compilation step, so writing the files is not really required in a successful build).
Of course, if the files are not written, and a run crashes, important information might be missing.
I am just raising this as a question (with no intention of working on it in the near future :) ): is a reduction of inodes count important for other sites?
There are also different (though more complex to implement) ways of reducing this, e.g. all .an files could be stores in a single file (sorting out concurrent writes from sub processes etc etc).
Contributor guide
Research direction
Start by reviewing the CI pipeline and Fab's handling of the listed preprocessed, .an, .mod, and .o files. Determine which artifacts are required after successful and failed builds, including concurrent subprocess behavior. Done would mean a concrete, agreed approach to reducing inode use without losing necessary recovery information.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fortran, python
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100