llnl / llnl/scr

Feature Request: Online Scalable Restart

Open
#614 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
108
Forks
37
PR merge metrics
No merged PRs in 30d

Description

[Fenix](https://github.com/sandialabs/Fenix) is a fault tolerance tool for online process recovery for MPI applications - meaning lost MPI ranks can be recovered without closing out the MPI session and returning to the shell. We've been working with the [LAMMPS](https://github.com/lammps/lammps) app and we're finalizing adding support for online recovery of LAMMPS. Fenix has its own purely in-memory checkpointing mechanisms, but LAMMPS' current checkpoint design is very tied to file writes and that makes it difficult to use our existing checkpoint API.

We could modify the Fenix checkpoint API (and might also do so), but I think SCR is pretty interesting and it supports a few other features that Fenix does not. So, it would be great to be able to support a Fenix+SCR model that can use the SCR scalable restart pattern. The Fenix team would be happy to help out in designing/implementing this functionality!

Is this something of interest to y'all? How complex do you expect it would be to support this in SCR?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing SCR's checkpoint API and the Fenix and LAMMPS integration described in the issue. Determine how online recovery and Fenix's in-memory checkpointing could fit SCR's scalable restart pattern. Done requires an agreed design, implementation scope, and maintainer confirmation of the relevant entry points.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.