resque / resque/resque-scheduler

Delayed jobs not executing

Open
#657 8 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Ruby
Stars
1.7k
Forks
477
PR merge metrics
No merged PRs in 30d

Description

Hi all!

First of all: thanks for this great project!

I've got resque (1.27.4) and scheduler (4.3.1) up and running with Rails (5.2.1). General background jobs are working fine, and it seemed delayed jobs were too. They are being added to the delayed queue (I can verify through the resque-web UI) and are correctly removed from the UI when the time comes. The only thing is that they actually never end up being executed. When looking through the logs, there's no info whatsoever. I also must note that I've tried it with different jobs (I've created a job with only 1 log as a test) and have tried it with ActiveJob as well as standalone Resque. When I schedule a job in the past, it does get executed (same if I hit 'execute now' in the UI)

I'm not entirely sure what information I should provide, so for completeness I'll provide my complete resque rake file.

require 'resque/tasks'
require 'resque/scheduler/tasks'


# Start a worker with proper env vars and output redirection
def run_worker(queue, count = 1)
  puts "Starting #{count} worker(s) with QUEUE: #{queue}"
  ops = {:pgroup => true, :err => [(Rails.root + "log/workers_error.log").to_s, "a"], 
                          :out => [(Rails.root + "log/workers.log").to_s, "a"]}
  env_vars = {"QUEUE" => queue.to_s}
  count.times {
    ## Using Kernel.spawn and Process.detach because regular system() call would
    ## cause the processes to quit when capistrano finishes
    pid = spawn(env_vars, "rake resque:work", ops)
    Process.detach(pid)
  }
end

# Start a scheduler, requires resque_scheduler >= 2.0.0.f
def run_scheduler
  puts "Starting resque scheduler"
  env_vars = {
    "BACKGROUND" => "1",
    "PIDFILE" => (Rails.root + "tmp/pids/resque_scheduler.pid").to_s,
    "VERBOSE" => "1"
  }
  ops = {:pgroup => true, :err => [(Rails.root + "log/scheduler_error.log").to_s, "a"],
                          :out => [(Rails.root + "log/scheduler.log").to_s, "a"]}
  pid = spawn(env_vars, "rake environment resque:scheduler", ops)
  Process.detach(pid)
end

namespace :resque do
  task :setup => :environment

  desc "Restart running workers"
  task :restart_workers => :environment do
    Rake::Task['resque:stop_workers'].invoke
    Rake::Task['resque:start_workers'].invoke
  end
  
  desc "Quit running workers"
  task :stop_workers => :environment do
    pids = Array.new
    Resque.workers.each do |worker|
      pids.concat(worker.worker_pids)
    end
    if pids.empty?
      puts "No workers to kill"
    else
      syscmd = "kill -s QUIT #{pids.join(' ')}"
      puts "Running syscmd: #{syscmd}"
      system(syscmd)
    end
  end
  
  desc "Start workers"
  task :start_workers => :environment do
    run_worker("*", 2)
    run_worker("high", 1)
  end

  desc "Restart scheduler"
  task :restart_scheduler => :environment do
    Rake::Task['resque:stop_scheduler'].invoke
    Rake::Task['resque:start_scheduler'].invoke
  end

  desc "Quit scheduler"
  task :stop_scheduler => :environment do
    pidfile = Rails.root + "tmp/pids/resque_scheduler.pid"
    if !File.exists?(pidfile)
      puts "Scheduler not running"
    else
      pid = File.read(pidfile).to_i
      syscmd = "kill -s QUIT #{pid}"
      puts "Running syscmd: #{syscmd}"
      system(syscmd)
      FileUtils.rm_f(pidfile)
    end
  end

  desc "Start scheduler"
  task :start_scheduler => :environment do
    run_scheduler
  end

  desc "Reload schedule"
  task :reload_schedule => :environment do
    pidfile = Rails.root + "tmp/pids/resque_scheduler.pid"

    if !File.exists?(pidfile)
      puts "Scheduler not running"
    else
      pid = File.read(pidfile).to_i
      syscmd = "kill -s USR2 #{pid}"
      puts "Running syscmd: #{syscmd}"
      system(syscmd)
    end
  end
end

My config file actually has nothing in it, since I'm not using the recurring feature yet.

I understand this question might seem a little vague, but I'd greatly appreciate it if someone could point me in the right direction. I've been looking through various docs, issues & blogs and can't seem to find anyone struggling with the same issues, which leads me to believe I'm overlooking something rather simple. If I'm missing some info, do tell. Thanks in advance!

Cheers!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the provided Resque rake file, especially the worker and scheduler process definitions, and inspect scheduler_error.log, scheduler.log, and workers.log alongside the resque-web delayed queue. Compare a job scheduled for the past with one scheduled for the future; done means future delayed jobs are handed to workers and their execution appears in the worker logs.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.