MESAHub / MESAHub/mesa

Reduce IO operations by using flush instead of repeated open/close.

Open
#169 0 comments 0 reactions 0 assignees View on GitHub
rainy day
Dominant language
Fortran
Stars
249
Forks
82
Avg merge
1d 3h
Merged PRs (30d)
3

Description

This was suggested by Nick Carriero of the Flatiron Scientific Computing Core. Nick wrote to me with:

> One quick thing to do would be to use flush rather than close, if the goal is to ensure data is written to disk. The metadata ops involved in closing and then reopening a file are much "heavier" than just reading from or writing to a file.

This came up because I was running many jobs on a cluster and the number of ops became an issue. Nick looked into it, noticed that 3/7 of those ops were involved in opening and re-closing `history.data`, `profiles.index`, and `pgstar.dat`, each of which gets opened then closed once per step.

His suggestion is to open these files once at the start of a run, then use the Fortran `FLUSH` intrinsic after our write statements are all done to ensure that all data is written to disk. This can happen once per step, and we just close the file handle at the very end.

I don't think this is urgent, and I have a workaround for my use case (just locating all of the files on a local scratch system then copying them back when the run finishes), but I didn't want to forget it because it seems like a good idea for a rainy day.

Contributor guide

Open the contributing guide

Research direction

Find the Fortran code that opens and closes history.data, profiles.index, and pgstar.dat, then trace when those operations occur during each step. Keep the files open for the run, flush them after each step's writes, and close the handles at the end; verify that output remains complete and usable.

Written by the indexing model from the issue text.

Assessment

Tech stack
fortran
Domain
hpc, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.