Can we auto-retry/recover lost runs?
Open
Nobody has claimed this yet.
discussion
- Dominant language
- Elixir
- Stars
- 296
- Forks
- 86
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 50
Description
As a user, lost runs sometimes feel like a punishment. I didn't even do anything but the app lost my run.
Given that it is not the user's fault, is there something the app can do to auto-recover?
- Can we auto-retry at least 3 times before calling it
lost? - Can we auto-detect when the db is busy and possibly delay claiming of runs?
- Suppose the
workercrashes and loses runs, can we add a feature toresumethe runs? The currentlightning <-> workerconnection is very strict, but I think if theworkercan detect that it lost connection to lightning, then it can rejoin theRunChanneland continue processing.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the worker-to-Lightning RunChannel connection and the existing handling of lost runs. Compare the three proposed directions—retrying, delaying claims when the database is busy, and resuming after a worker disconnect—to determine the intended scope. Done requires an agreed recovery behavior and tests or other verification for it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elixir
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100