ocurrent / ocurrent/ocluster

Kick off workers that don't respond to metrics requests

Open
#110 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
OCaml
Stars
42
Forks
20
PR merge metrics
No merged PRs in 30d

Description

Sometimes workers get stuck in uninterruptable waits (due to btrfs bugs). In that case, the TCP connection remains alive but the worker never does any work. The scheduler should detect this (e.g. by a timeout when collecting metrics) and mark the worker as paused (or restarting), so that queued jobs for that worker get reassigned.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the scheduler's metrics-collection path and the worker state transitions for paused or restarting workers. Trace how queued jobs are reassigned when a worker becomes unavailable; done means a nonresponsive metrics request is detected despite a live TCP connection, the worker is marked unavailable, and its queued jobs can be reassigned.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.