Update worker to be able to reference a local scheduler

Open
#1,847 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
rust
Domain
backend, build-system, distributed-systems

Research direction

Start by tracing how worker_api_endpoint.uri is parsed and how scheduler names are resolved. Define the behavior for a local:// URI, including how a worker finds the named local scheduler without exposing a gRPC port. Done means local scheduler references work for workers while existing URI schemes continue to work; add or update tests for both paths.

Written by the indexing model from the issue text.

Description

If using a Redis scheduler, there's very little need to have a connection between the worker and the scheduler pool. Instead, each worker can simply run its own internal scheduler which assigns work to its own workers. This is also useful for testing on a local machine rather than requiring a loopback address.

Although this is already supported by simply using a loopback, it would be nice if it was supported internally without having to expose the gRPC port.

The proposal is to update the worker_api_endpoint.uri to support a scheme "local://" which references the local scheduler by name.

Dominant language
Rust
Stars
1.6k
Forks
244
Avg merge
1d 16h
Merged PRs (30d)
54

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TraceMachina/nativelink

All issues in TraceMachina/nativelink

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.