resque / resque/resque-scheduler
Delayed jobs not executing
Nobody has claimed this yet.
- Dominant language
- Ruby
- Stars
- 1.7k
- Forks
- 477
- PR merge metrics
- No merged PRs in 30d
Description
Hi all!
First of all: thanks for this great project!
I've got resque (1.27.4) and scheduler (4.3.1) up and running with Rails (5.2.1). General background jobs are working fine, and it seemed delayed jobs were too. They are being added to the delayed queue (I can verify through the resque-web UI) and are correctly removed from the UI when the time comes. The only thing is that they actually never end up being executed. When looking through the logs, there's no info whatsoever. I also must note that I've tried it with different jobs (I've created a job with only 1 log as a test) and have tried it with ActiveJob as well as standalone Resque. When I schedule a job in the past, it does get executed (same if I hit 'execute now' in the UI)
I'm not entirely sure what information I should provide, so for completeness I'll provide my complete resque rake file.
require 'resque/tasks'
require 'resque/scheduler/tasks'
# Start a worker with proper env vars and output redirection
def run_worker(queue, count = 1)
puts "Starting #{count} worker(s) with QUEUE: #{queue}"
ops = {:pgroup => true, :err => [(Rails.root + "log/workers_error.log").to_s, "a"],
:out => [(Rails.root + "log/workers.log").to_s, "a"]}
env_vars = {"QUEUE" => queue.to_s}
count.times {
## Using Kernel.spawn and Process.detach because regular system() call would
## cause the processes to quit when capistrano finishes
pid = spawn(env_vars, "rake resque:work", ops)
Process.detach(pid)
}
end
# Start a scheduler, requires resque_scheduler >= 2.0.0.f
def run_scheduler
puts "Starting resque scheduler"
env_vars = {
"BACKGROUND" => "1",
"PIDFILE" => (Rails.root + "tmp/pids/resque_scheduler.pid").to_s,
"VERBOSE" => "1"
}
ops = {:pgroup => true, :err => [(Rails.root + "log/scheduler_error.log").to_s, "a"],
:out => [(Rails.root + "log/scheduler.log").to_s, "a"]}
pid = spawn(env_vars, "rake environment resque:scheduler", ops)
Process.detach(pid)
end
namespace :resque do
task :setup => :environment
desc "Restart running workers"
task :restart_workers => :environment do
Rake::Task['resque:stop_workers'].invoke
Rake::Task['resque:start_workers'].invoke
end
desc "Quit running workers"
task :stop_workers => :environment do
pids = Array.new
Resque.workers.each do |worker|
pids.concat(worker.worker_pids)
end
if pids.empty?
puts "No workers to kill"
else
syscmd = "kill -s QUIT #{pids.join(' ')}"
puts "Running syscmd: #{syscmd}"
system(syscmd)
end
end
desc "Start workers"
task :start_workers => :environment do
run_worker("*", 2)
run_worker("high", 1)
end
desc "Restart scheduler"
task :restart_scheduler => :environment do
Rake::Task['resque:stop_scheduler'].invoke
Rake::Task['resque:start_scheduler'].invoke
end
desc "Quit scheduler"
task :stop_scheduler => :environment do
pidfile = Rails.root + "tmp/pids/resque_scheduler.pid"
if !File.exists?(pidfile)
puts "Scheduler not running"
else
pid = File.read(pidfile).to_i
syscmd = "kill -s QUIT #{pid}"
puts "Running syscmd: #{syscmd}"
system(syscmd)
FileUtils.rm_f(pidfile)
end
end
desc "Start scheduler"
task :start_scheduler => :environment do
run_scheduler
end
desc "Reload schedule"
task :reload_schedule => :environment do
pidfile = Rails.root + "tmp/pids/resque_scheduler.pid"
if !File.exists?(pidfile)
puts "Scheduler not running"
else
pid = File.read(pidfile).to_i
syscmd = "kill -s USR2 #{pid}"
puts "Running syscmd: #{syscmd}"
system(syscmd)
end
end
end
My config file actually has nothing in it, since I'm not using the recurring feature yet.
I understand this question might seem a little vague, but I'd greatly appreciate it if someone could point me in the right direction. I've been looking through various docs, issues & blogs and can't seem to find anyone struggling with the same issues, which leads me to believe I'm overlooking something rather simple. If I'm missing some info, do tell. Thanks in advance!
Cheers!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided Resque rake file, especially the worker and scheduler process definitions, and inspect scheduler_error.log, scheduler.log, and workers.log alongside the resque-web delayed queue. Compare a job scheduled for the past with one scheduled for the future; done means future delayed jobs are handed to workers and their execution appears in the worker logs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100