OpenPipe / OpenPipe/ART

Allow running multiple `art.LocalBackend` sessions on different processes

Open
#396 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
10.8k
Forks
989
Avg merge
6h 29m
Merged PRs (30d)
85

Description

Currently it's impossible to launch several LocalBackend(in_process=False) sessions simultaneously on different processes.

The current implementation of the LocalBackend performs: "pkill -9 model-service" which kills the "model-service" systemwide. As result, if I launched a LocalBackend session on the first process and started training, and then on another process I launch another LocalBackend session, it kills the run of the first session.

My use case is that I run several training scripts which all launch a LocalBackend automatically, each with their own model and setup, each with their dedicated GPUs.

Also, given the recent improvements made to closing LocalBackend (see #126), is running "pkill -9 model-service" even still necessary?

Note: it's possible to launch several sessions of LocalBackend(in_process=True) but at that case they suffer from performance penalties, since Unsloth optimizations are not applied (i.e 2-3x slowdown compared to LocalBackend(in_process=False)).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the LocalBackend implementation, focusing on the in_process=False startup and cleanup path that runs pkill -9 model-service. Reproduce two separate processes with dedicated sessions, then verify that stopping or starting one session does not terminate the other and that both training runs continue independently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.