Azure / Azure/Azure-DataFactory

Running R scripts seems to be deprecated

Open
#375 5 comments 1 reaction 0 assignees View on GitHub
Dominant language
PowerShell
Stars
529
Forks
623
PR merge metrics
No merged PRs in 30d

Description

While following the steps outlined in the tutorial: https://github.com/Azure/Azure-DataFactory/blob/main/SamplesV2/tutorial-run-r-batch-azure-data-factory/tutorial-run-r-batch-azure-data-factory.md , I encountered an error debugging the pipeline:

> {"errorCategory":0,"code":"CommandProgramNotFound","message":"The specified command program is not found","details":[{"Name":"Message","Value":"The system cannot find the file specified."}]}

It seems the R bin directory is not in the PATH of VMs instantiated from the dsvm-win-2019 offer. Next, I tried changing the command to point directly to the executable:

`cmd /C "C:\Program Files\R\R-4.1.2\bin\Rscript.exe" test.r`

This results in the following error from the Batch Service:

> Warning in install.packages("rAzureBatch") : 'lib = "C:/Program Files/R/R-4.1.2/library"' is not writable
> Error in install.packages("rAzureBatch") : unable to install packages
> Execution halted

It seems the default library directory is not writable by the batch service. I redirected the library to the working folder of the task:

`cmd /C set "PATH=C:\Program Files\R\R-4.1.2\bin;%PATH%" && set "R_LIBS=%AZ_BATCH_TASK_WORKING_DIR%" && Rscript.exe test.r`

And got this error:

> Error in contrib.url(repos, "source") : trying to use CRAN without setting a mirror
> Calls: install.packages -> contrib.url
> Execution halted

I dropped an .Rprofile into the storage account alongside my R script:

`options(repos=structure(c(CRAN="http://cran.r-project.org")))`

And got this error:

> Error in library(rAzureBatch) : there is no package called 'rAzureBatch'

It seems this package is not longer maintained: https://github.com/Azure/rAzureBatch

Therefore, I assume running R scripts via Batch Service through Azure Data Factory is no longer supported. If so, should tutorial-run-r-batch-azure-data-factory.md be updated?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.