mlr-org / mlr-org/batchtools

runOSCommand throws error with R 4.0 (missing shQuote)

Open
#264 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
184
Forks
53
Avg merge
7d 2h
Merged PRs (30d)
1

Description

Hi,
I'm working on a debian server and try to use makeClusterFunctionsSSH(...) to distribute jobs to other debian servers. It throws the following error:
syntax error near unexpected token `('
The error is caused by runOSCommand(...).
In https://github.com/mllg/batchtools/commit/75a5b701341c6937c779ce395fe9367dbee99c15 you removed the shQuote for R 4.0 and higher. But as far as I was able to reproduce the error, this shQuote is still needed with R 4.0.2.

MWE directly built from the code of Worker and runOSCommand:
(you have to set the nodename variable)

library(batchtools)
### error:

nodename = "XXXXX"
sys.args = c("-e", shQuote("message(\"[bt] --BOF--\\n\", \"[bt] \", system.file(\"bin/linux-helper\", package = \"batchtools\"), \"\\n[bt] --EOF--\\n\")"))
sys.cmd = "Rscript"
checkmate::assertCharacter(sys.cmd, any.missing = FALSE, len = 1L)
checkmate::assertCharacter(sys.args, any.missing = FALSE)
checkmate::assertString(nodename, min.chars = 1L)
command = sprintf("%s %s", sys.cmd, stringi::stri_flatten(sys.args, " "))
command = stringi::stri_replace_all_fixed(command, "\\$", "$")
sys.args = c("-q", nodename, command)
sys.cmd = "ssh"
system2(command = sys.cmd, args = sys.args, stdin = "", stdout = TRUE, stderr = TRUE, wait = TRUE)
#> Warning in system2(command = sys.cmd, args = sys.args, stdin = "", stdout =
#> TRUE, : running command ''ssh' -q XXXXX
#> Rscript -e 'message("[bt] --BOF--\n", "[bt] ", system.file("bin/linux-helper",
#> package = "batchtools"), "\n[bt] --EOF--\n")' 2>&1' had status 1
#> [1] "bash: -c: line 0: syntax error near unexpected token `('"                                                                                                   
#> [2] "bash: -c: line 0: `Rscript -e message(\"[bt] --BOF--\\n\", \"[bt] \", system.file(\"bin/linux-helper\", package = \"batchtools\"), \"\\n[bt] --EOF--\\n\")'"
#> attr(,"status")
#> [1] 1


#### works:

command = shQuote(command)
sys.args = c("-q", nodename, command)
sys.cmd = "ssh"
system2(command = sys.cmd, args = sys.args, stdin = "", stdout = TRUE, stderr = TRUE, wait = TRUE)
#> [1] "[bt] --BOF--"                                                               
#> [2] "[bt] /home/pa/R/x86_64-pc-linux-gnu-library/4.0/batchtools/bin/linux-helper"
#> [3] "[bt] --EOF--"                                                               
#> [4] ""

Created on 2020-07-17 by the reprex package (v0.3.0)

Can you reproduce this?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with runOSCommand and the Worker command construction described in the issue, then run the provided MWE on Debian with R 4.0.2 to compare the failing and shQuote-wrapped commands. Done means makeClusterFunctionsSSH can execute the remote Rscript command without the shell syntax error.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.