hydrusnetwork / hydrusnetwork/hydrus

Use Work Stealing instead of idling workers threads

Open
#809 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature-request
Dominant language
Python
Stars
3.2k
Forks
207
PR merge metrics
No merged PRs in 30d

Description

If THREADCallToThread wakes up right now and finds no work in its queue, it basically just goes to sleep and eats the cost of a context switch. Since work put in HydrusController.CallToThread(fn,:callable,*args,**kwargs) is either assigned to a worker with a queue that is empty or at random, this means that the only time a worker is checked to see if it is idle is put in the thread of the caller. Suppose for the sake of argument that over the program's execution, a number of long running tasks have accumulated in a particular worker's queue, and since it is spending a comparatively long amount of time waiting on completion it cannot process the other tasks, even if as the other threads have finished their work and are mostly taking a nap.

What should happen instead is either that

  • A: Workers are redesigned to pull work from a shared queue in the thread pool, or
  • B: Workers that have no worl inspect the queues of other threads in the pool, and steal the next available element.
    def run( self ):
        
        try:
            
            while True:
                
                #If it appears empty suspend for 10 seconds, then check again.
                #Queue.empty uses the queue count outside of a critical section, so it is heuristic.
                #It is safe since we are the only ones who can empty this queue, so if we see an empty queue,
                #then immediately get suspended and an element gets added at worst we will sleep 10 seconds, and then see a non empty queue.
                while self._queue.empty():
                    
                    CheckIfThreadShuttingDown()
                    
                    self._event.wait( 10.0 )
                    
                    self._event.clear()

This is the code as it is at present, the thread just waits for work, even if there is plenty of work it's just not in its queue.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at THREADCallToThread.run and HydrusController.CallToThread(fn,:callable,*args,**kwargs), including the queue and event behavior shown in the issue. Compare the shared-queue and queue-stealing approaches before choosing a design. Done means idle workers can take available work from elsewhere in the pool instead of sleeping while another worker's queue remains busy.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.