Using Python subprocess.Popen in Rmd Failed
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 1.8k
- Forks
- 349
- Avg merge
- 5h 13m
- Merged PRs (30d)
- 4
Description
I am using Rmd and reticulate python eval function, which aims at opening a file via Python subprocess.Popen command, however, it seems sub-threading is not available. BTW, it works very well in the terminal.
my code is
import subprocess
def getFilefromHDFS(path):
args_list = ['hadoop', 'fs', '-ls', path]
proc = subprocess.Popen(args_list, stdout=subprocess.PIPE,stderr=subprocess.PIPE)
proc.communicate()
print proc.returncode
getFilefromHDFS("/user/hive/warehouse/ai.db/xxx/")
The error is :
OSError: [Errno 2] No such file or directory
here is my session info:
R version 3.5.1 (2018-07-02)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS
Matrix products: default
BLAS: /usr/local/lib/R/lib/libRblas.so
LAPACK: /usr/local/lib/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods
[7] base
other attached packages:
[1] sp_1.3-1 leaflet_1.1.0.9000 crosstalk_1.0.0
[4] mlflow_0.7.0 bindrcpp_0.2.2 dplyr_0.7.8
loaded via a namespace (and not attached):
[1] Rcpp_1.0.0 lattice_0.20-35 forge_0.1.0
[4] tidyr_0.8.2 png_0.1-7 ps_1.1.0
[7] utf8_1.1.4 assertthat_0.2.0 rprojroot_1.3-2
[10] digest_0.6.16 packrat_0.4.9-3 showimage_1.0.0
[13] mime_0.5 plyr_1.8.4 R6_2.3.0
[16] backports_1.1.2 evaluate_0.11 httr_1.3.1
[19] pillar_1.3.0 rlang_0.3.0.1 lazyeval_0.2.1
[22] rematch_1.0.1 curl_3.2 rstudioapi_0.7
[25] callr_3.0.0 Matrix_1.2-14 DT_0.5
[28] reticulate_1.10 rmarkdown_1.10 readr_1.1.1
[31] stringr_1.3.1 htmlwidgets_1.3 munsell_0.5.0
[34] shiny_1.1.0 compiler_3.5.1 pingr_1.1.2
[37] httpuv_1.4.5 pkgconfig_2.0.2 base64enc_0.1-3
[40] htmltools_0.3.6 openssl_1.0.2 tidyselect_0.2.5
[43] tibble_1.4.2 shinytest_1.3.0 fansi_0.3.0
[46] crayon_1.3.4 withr_2.1.2 later_0.7.4
[49] grid_3.5.1 jsonlite_1.5 xtable_1.8-3
[52] git2r_0.23.0 magrittr_1.5 scales_1.0.0
[55] geohash_0.4.0 cli_1.0.0 stringi_1.2.4
[58] debugme_1.1.0 swagger_3.9.2 webdriver_1.0.5
[61] fs_1.2.6 promises_1.0.1 testthat_2.0.1
[64] parsedate_1.1.3 tools_3.5.1 glue_1.3.0
[67] purrr_0.2.5 hms_0.4.2 processx_3.2.0
[70] yaml_2.2.0 colorspace_1.3-2 knitr_1.20
[73] bindr_0.1.1
need help,
Thanks
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Rmd call through reticulate's Python eval with the shown subprocess.Popen example, then compare it with the terminal execution and the supplied session information. The issue does not name project files or tests; done means identifying the cause of the OSError and establishing a working or clearly documented subprocess behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, r
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100