mlr-org / mlr-org/batchtools

TORQUE trouble

Open
#149 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
184
Forks
53
Avg merge
7d 2h
Merged PRs (30d)
1

Description

I am having trouble running batchtools jobs on a local installation of TORQUE on Ubuntu 16.04. I think TORQUE is working because the following test.pbs produces the expected output.

#PBS -N test
#PBS -l nodes=1:ppn=1
#PBS -l walltime=0:01:00

cd $PBS_O_WORKDIR
touch done.txt
echo "done"

However, all my jobs hang in the E state. For example, the following R script waits indefinitely.

library("batchtools")
cf <- makeClusterFunctionsTORQUE("torque.tmpl") 
reg <- makeRegistry(NA)
reg$cluster.functions <- cf
batchMap(fun = identity, x = 1:4)
submitJobs()
waitForJobs() # waits here indefinitely
reduceResultsList() # not reached

In my case, the console message of wait_for_jobs()

Waiting (S:4 R:4 D:0 E:0) [-------------------]   0% eta:  ?s

does not match qstat, which shows jobs hanging in the E state.

Job id                    Name             User            Time Use S Queue
------------------------- ---------------- --------------- -------- - -----
98.localhost              ...8d7bd98804b04 wlandau         00:00:00 E batch          
99.localhost              ...fcce12fedcace wlandau         00:00:00 E batch          
100.localhost             ...dc63017b37ac6 wlandau         00:00:00 E batch          
101.localhost             ...b060e52879b8e wlandau         00:00:00 E batch 

I am using the @HenrikBengtsson's torque.tmpl from future.batchtools.

Related: see my Stack Overflow post here and HenrikBengtsson/future.batchtools#12.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported R script, especially makeClusterFunctionsTORQUE, submitJobs, and waitForJobs, and inspect the referenced torque.tmpl from future.batchtools. Reproduce the four-job example alongside qstat, then trace how batchtools reads job states. Done means the jobs no longer remain in E and waitForJobs reports their actual completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
distributed-systems, hpc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.