PolicyEngine / PolicyEngine/policyengine-api

Add checkpointing for long-running computations

Open
#1,254 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
18
Forks
33
Avg merge
23h 40m
Merged PRs (30d)
22

Description

gcloud recommends that

For long-running computations, periodically create checkpoints so that you can resume from that state.

Of course, these checkpoints need to be stored using a cloud service (not locally, see #1253).

The lack of checkpoints and frequent restarts may explain why some long-running computations seem to never end. For instance, I started this economic impact calculation two hours ago and it is still running. It could also be the case that the computation is just that long.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by locating how long-running computations are initiated and where cloud persistence is configured; done means computations can periodically save checkpoints remotely and resume after a restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.