rsession pegs CPU at 100% and hangs without error
- Dominant language
- Python
- Stars
- 20.4k
- Forks
- 4.6k
- Avg merge
- 19h 52m
- Merged PRs (30d)
- 1
Description
## Problem Overview
My R session invariably hangs with the rsession %CPU at 100% some time after I run predict. However, the predict function runs properly and will execute the prediction and any number of other functions after that point until I save an object using readr::write_csv, switch to a different project, save the script or some other write to disk type of function. Once I use the predict function the rsession invariably hangs at some point. This does not occur unless prophet::predict is called. It occurs if the prophet package is loaded via library(prophet) or called prophet::predict.
It may be important to note that I am connecting to an r session on a google cloud compute engine using rstudio-server.
## Reproducible Example
Here's an example that exhibits the behavior on my machine without fail:
```
library(prophet)
library(tidyverse)
df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_peyton_manning.csv')
m <- prophet(df)
future <- make_future_dataframe(m, periods = 365)
forecast <- predict(m, future) # this will execute properly
write_csv(df, 'prophet_test_file.csv') # session hangs at this point
```
## Session Info
```
R version 4.2.0 (2022-04-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS
Matrix products: default
BLAS/LAPACK: /opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64_lin/libmkl_rt.so
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
[4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8
[7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] forcats_0.5.1 stringr_1.4.0 dplyr_1.0.9 purrr_0.3.4
[5] readr_2.1.2 tidyr_1.2.0 tibble_3.1.8 ggplot2_3.3.6
[9] tidyverse_1.3.2 prophet_1.0 rlang_1.0.4 Rcpp_1.0.9
loaded via a namespace (and not attached):
[1] cellranger_1.1.0 pillar_1.8.0 compiler_4.2.0
[4] dbplyr_2.2.1 tools_4.2.0 lubridate_1.8.0
[7] jsonlite_1.8.0 googledrive_2.0.0 lifecycle_1.0.1
[10] gargle_1.2.0 gtable_0.3.0 pkgconfig_2.0.3
[13] reprex_2.0.1 DBI_1.1.3 cli_3.3.0
[16] rstudioapi_0.13 haven_2.5.0 xml2_1.3.3
[19] withr_2.5.0 httr_1.4.3 hms_1.1.1
[22] fs_1.5.2 generics_0.1.3 vctrs_0.4.1
[25] googlesheets4_1.0.0 grid_4.2.0 tidyselect_1.1.2
[28] glue_1.6.2 R6_2.5.1 fansi_1.0.3
[31] readxl_1.4.0 tzdb_0.3.0 modelr_0.1.8
[34] magrittr_2.0.3 ellipsis_0.3.2 backports_1.4.1
[37] scales_1.2.0 codetools_0.2-18 rvest_1.0.2
[40] assertthat_0.2.1 colorspace_2.0-3 utf8_1.2.2
[43] stringi_1.7.8 RcppParallel_5.1.5 munsell_0.5.0
[46] broom_1.0.0 crayon_1.5.1
```
## Rstudio Server
```
RStudio 2022.07.0+548 "Spotted Wakerobin" Release (34ea3031089fa4e38738a9256d6fa6d70629c822, 2022-07-06) for Ubuntu Bionic
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
```
## perf top readout when rsession is hung
```
Samples: 33K of event 'cpu-clock:pppH', 4000 Hz, Event count (approx.): 6900362281 lost: 0/0 drop: 0/0
Children Self Command Parent symbol Shared Object Pid:Command
+ 92.23% 33.40% rsession [other] /lib/x86_64-linux-gnu/libc-2.27.so 3918:rsession
+ 38.20% 38.20% rsession [other] /proc/kcore 3918:rsession
+ 23.64% 23.53% rsession [other] /opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin/libiomp5.so 3918:rsession
+ 2.13% 1.91% perf [other] /usr/lib/linux-gcp-5.4-tools-5.4.0-1083/perf 10593:perf
+ 2.07% 0.02% perf [other] /lib/x86_64-linux-gnu/libpthread-2.27.so 10593:perf
+ 2.06% 0.79% perf [other] /lib/x86_64-linux-gnu/libc-2.27.so 10593:perf
1.30% 1.30% perf [other] /proc/kcore 10593:perf
+ 0.91% 0.00% rsession [other] [unknown] 3918:rsession
0.40% 0.40% swapper [other] /proc/kcore 0:swapper
+ 0.15% 0.15% perf do_page_fault /proc/kcore 10593:perf
0.15% 0.10% perf [other] /lib/x86_64-linux-gnu/libc-2.27.so 10581:perf
0.05% 0.04% perf [other] /usr/lib/linux-gcp-5.4-tools-5.4.0-1083/perf 10581:perf
0.03% 0.01% perf-top-UI [other] /usr/lib/linux-gcp-5.4-tools-5.4.0-1083/perf 10594:perf-top-UI
0.02% 0.02% perf-top-UI [other] /lib/x86_64-linux-gnu/libslang.so.2.3.1 10594:perf-top-UI
0.02% 0.02% rcu_sched [other] /proc/kcore 11:rcu_sched
0.01% 0.00% perf [other] [unknown] 10581:perf
0.01% 0.01% rserver [other] /usr/lib/rstudio-server/bin/rserver 1473:rserver
0.01% 0.01% perf do_page_fault /proc/kcore 10581:perf
0.01% 0.01% rsession [other] /proc/kcore 3921:rsession
0.01% 0.01% rserver [other] /usr/lib/rstudio-server/bin/rserver 1472:rserver
0.01% 0.01% perf [other] /proc/kcore 10581:perf
0.01% 0.01% rserver [other] /lib/x86_64-linux-gnu/libc-2.27.so 1473:rserver
0.01% 0.00% perf [other] [unknown] 10593:perf
0.01% 0.00% rserver [other] [unknown] 1472:rserver
0.01% 0.01% rserver [other] /lib/x86_64-linux-gnu/libz.so.1.2.11 1472:rserver
0.01% 0.01% rserver [other] /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 1472:rserver
0.01% 0.01% rserver [other] /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 1473:rserver
0.01% 0.01% rsession [other] /usr/lib/rstudio-server/bin/rsession 3921:rsession
0.01% 0.01% perf [other] /usr/lib/x86_64-linux-gnu/libelf-0.170.so 10593:perf
0.01% 0.01% perf-top-UI [other] /lib/x86_64-linux-gnu/libc-2.27.so 10594:perf-top-UI
0.00% 0.00% kworker/5:4-mm_ [other] /proc/kcore 8344:kworker/5:4-mm_
0.00% 0.00% google_guest_ag [other] /usr/bin/google_guest_agent 1805:google_guest_ag
0.00% 0.00% google_guest_ag [other] [unknown] 1805:google_guest_ag
0.00% 0.00% rsession [other] [unknown] 3921:rsession
0.00% 0.00% rsession [other] /lib/x86_64-linux-gnu/libpthread-2.27.so 3921:rsession
0.00% 0.00% cron [other] /proc/kcore 2000:cron
0.00% 0.00% kworker/2:0-eve [other] /proc/kcore 10255:kworker/2:0-eve
0.00% 0.00% snapd [other] /proc/kcore 1758:snapd
0.00% 0.00% sshd [other] /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 9691:sshd
0.00% 0.00% perf [other] /lib/x86_64-linux-gnu/libslang.so.2.3.1 10593:perf
0.00% 0.00% rserver [other] /usr/lib/rstudio-server/bin/rserver 1470:rserver
0.00% 0.00% rserver [other] /lib/x86_64-linux-gnu/libc-2.27.so 1470:rserver
0.00% 0.00% rserver [other] /proc/kcore 1472:rserver
0.00% 0.00% sshd [other] /proc/kcore 9691:sshd
```
I'm sure detailed logs showing more specifically what the rsession is hanging on would be helpful but I have not been able to find any instruction on how to obtain those logs from within a google cloud vm. I will gladly generate additional logs if I could get pointed in the right direction of how to get them.
Thank you
Contributor guide
Assessment
This issue has not been assessed yet.