facebook / facebook/prophet

rsession pegs CPU at 100% and hangs without error

Open
#2,237 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
20.4k
Forks
4.6k
Avg merge
19h 52m
Merged PRs (30d)
1

Description

## Problem Overview
My R session invariably hangs with the rsession %CPU at 100% some time after I run predict. However, the predict function runs properly and will execute the prediction and any number of other functions after that point until I save an object using readr::write_csv, switch to a different project, save the script or some other write to disk type of function. Once I use the predict function the rsession invariably hangs at some point. This does not occur unless prophet::predict is called. It occurs if the prophet package is loaded via library(prophet) or called prophet::predict.

It may be important to note that I am connecting to an r session on a google cloud compute engine using rstudio-server.

## Reproducible Example
Here's an example that exhibits the behavior on my machine without fail:
```
library(prophet)
library(tidyverse)

df <- read.csv('https://raw.githubusercontent.com/facebook/prophet/main/examples/example_wp_log_peyton_manning.csv')
m <- prophet(df)
future <- make_future_dataframe(m, periods = 365)
forecast <- predict(m, future) # this will execute properly

write_csv(df, 'prophet_test_file.csv') # session hangs at this point
```
## Session Info

```
R version 4.2.0 (2022-04-22)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.6 LTS

Matrix products: default
BLAS/LAPACK: /opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64_lin/libmkl_rt.so

locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
[4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8
[7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] forcats_0.5.1 stringr_1.4.0 dplyr_1.0.9 purrr_0.3.4
[5] readr_2.1.2 tidyr_1.2.0 tibble_3.1.8 ggplot2_3.3.6
[9] tidyverse_1.3.2 prophet_1.0 rlang_1.0.4 Rcpp_1.0.9

loaded via a namespace (and not attached):
[1] cellranger_1.1.0 pillar_1.8.0 compiler_4.2.0
[4] dbplyr_2.2.1 tools_4.2.0 lubridate_1.8.0
[7] jsonlite_1.8.0 googledrive_2.0.0 lifecycle_1.0.1
[10] gargle_1.2.0 gtable_0.3.0 pkgconfig_2.0.3
[13] reprex_2.0.1 DBI_1.1.3 cli_3.3.0
[16] rstudioapi_0.13 haven_2.5.0 xml2_1.3.3
[19] withr_2.5.0 httr_1.4.3 hms_1.1.1
[22] fs_1.5.2 generics_0.1.3 vctrs_0.4.1
[25] googlesheets4_1.0.0 grid_4.2.0 tidyselect_1.1.2
[28] glue_1.6.2 R6_2.5.1 fansi_1.0.3
[31] readxl_1.4.0 tzdb_0.3.0 modelr_0.1.8
[34] magrittr_2.0.3 ellipsis_0.3.2 backports_1.4.1
[37] scales_1.2.0 codetools_0.2-18 rvest_1.0.2
[40] assertthat_0.2.1 colorspace_2.0-3 utf8_1.2.2
[43] stringi_1.7.8 RcppParallel_5.1.5 munsell_0.5.0
[46] broom_1.0.0 crayon_1.5.1
```

## Rstudio Server
```
RStudio 2022.07.0+548 "Spotted Wakerobin" Release (34ea3031089fa4e38738a9256d6fa6d70629c822, 2022-07-06) for Ubuntu Bionic
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
```

## perf top readout when rsession is hung
```
Samples: 33K of event 'cpu-clock:pppH', 4000 Hz, Event count (approx.): 6900362281 lost: 0/0 drop: 0/0
Children Self Command Parent symbol Shared Object Pid:Command
+ 92.23% 33.40% rsession [other] /lib/x86_64-linux-gnu/libc-2.27.so 3918:rsession
+ 38.20% 38.20% rsession [other] /proc/kcore 3918:rsession
+ 23.64% 23.53% rsession [other] /opt/intel/compilers_and_libraries_2018.2.199/linux/compiler/lib/intel64_lin/libiomp5.so 3918:rsession
+ 2.13% 1.91% perf [other] /usr/lib/linux-gcp-5.4-tools-5.4.0-1083/perf 10593:perf
+ 2.07% 0.02% perf [other] /lib/x86_64-linux-gnu/libpthread-2.27.so 10593:perf
+ 2.06% 0.79% perf [other] /lib/x86_64-linux-gnu/libc-2.27.so 10593:perf
1.30% 1.30% perf [other] /proc/kcore 10593:perf
+ 0.91% 0.00% rsession [other] [unknown] 3918:rsession
0.40% 0.40% swapper [other] /proc/kcore 0:swapper
+ 0.15% 0.15% perf do_page_fault /proc/kcore 10593:perf
0.15% 0.10% perf [other] /lib/x86_64-linux-gnu/libc-2.27.so 10581:perf
0.05% 0.04% perf [other] /usr/lib/linux-gcp-5.4-tools-5.4.0-1083/perf 10581:perf
0.03% 0.01% perf-top-UI [other] /usr/lib/linux-gcp-5.4-tools-5.4.0-1083/perf 10594:perf-top-UI
0.02% 0.02% perf-top-UI [other] /lib/x86_64-linux-gnu/libslang.so.2.3.1 10594:perf-top-UI
0.02% 0.02% rcu_sched [other] /proc/kcore 11:rcu_sched
0.01% 0.00% perf [other] [unknown] 10581:perf
0.01% 0.01% rserver [other] /usr/lib/rstudio-server/bin/rserver 1473:rserver
0.01% 0.01% perf do_page_fault /proc/kcore 10581:perf
0.01% 0.01% rsession [other] /proc/kcore 3921:rsession
0.01% 0.01% rserver [other] /usr/lib/rstudio-server/bin/rserver 1472:rserver
0.01% 0.01% perf [other] /proc/kcore 10581:perf
0.01% 0.01% rserver [other] /lib/x86_64-linux-gnu/libc-2.27.so 1473:rserver
0.01% 0.00% perf [other] [unknown] 10593:perf
0.01% 0.00% rserver [other] [unknown] 1472:rserver
0.01% 0.01% rserver [other] /lib/x86_64-linux-gnu/libz.so.1.2.11 1472:rserver
0.01% 0.01% rserver [other] /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 1472:rserver
0.01% 0.01% rserver [other] /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25 1473:rserver
0.01% 0.01% rsession [other] /usr/lib/rstudio-server/bin/rsession 3921:rsession
0.01% 0.01% perf [other] /usr/lib/x86_64-linux-gnu/libelf-0.170.so 10593:perf
0.01% 0.01% perf-top-UI [other] /lib/x86_64-linux-gnu/libc-2.27.so 10594:perf-top-UI
0.00% 0.00% kworker/5:4-mm_ [other] /proc/kcore 8344:kworker/5:4-mm_
0.00% 0.00% google_guest_ag [other] /usr/bin/google_guest_agent 1805:google_guest_ag
0.00% 0.00% google_guest_ag [other] [unknown] 1805:google_guest_ag
0.00% 0.00% rsession [other] [unknown] 3921:rsession
0.00% 0.00% rsession [other] /lib/x86_64-linux-gnu/libpthread-2.27.so 3921:rsession
0.00% 0.00% cron [other] /proc/kcore 2000:cron
0.00% 0.00% kworker/2:0-eve [other] /proc/kcore 10255:kworker/2:0-eve
0.00% 0.00% snapd [other] /proc/kcore 1758:snapd
0.00% 0.00% sshd [other] /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 9691:sshd
0.00% 0.00% perf [other] /lib/x86_64-linux-gnu/libslang.so.2.3.1 10593:perf
0.00% 0.00% rserver [other] /usr/lib/rstudio-server/bin/rserver 1470:rserver
0.00% 0.00% rserver [other] /lib/x86_64-linux-gnu/libc-2.27.so 1470:rserver
0.00% 0.00% rserver [other] /proc/kcore 1472:rserver
0.00% 0.00% sshd [other] /proc/kcore 9691:sshd
```

I'm sure detailed logs showing more specifically what the rsession is hanging on would be helpful but I have not been able to find any instruction on how to obtain those logs from within a google cloud vm. I will gladly generate additional logs if I could get pointed in the right direction of how to get them.

Thank you

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.