fread randomly crashes Rstudio (server) when in a loop

No source file or test is named. Start by reproducing the loop with the reported R 4.1.0, data.table 1.14.2, Ubuntu 20.04.3, and large TSV workload, then isolate why fread crashes intermittently; done means the crash cause is identified and the behavior is covered by a regression test.

Using Rstudio server with R 4.1.0 and data.table 1.14.2.

This is a hard one to reproduce, but I have about 1000 tsv files that I am reading and appending together in a loop. Each file has ~150k lines and 23 columns. For some reason it randomly crashes R when it's somewhere in the middle of the loop, and also at random points. At first, I used rbindlist(lapply(files,fread) or files %>% map(fread) and when it crashed I thought there was something wrong with the files themselves. But when I made a loop like:

out <- list()
for(i in 1:length(files)) {
  print(files[i])
  out[[i]] <- fread(files[i])
  print("done")
}

It would crash randomly in the loop when I run this multiple times.

Concatenating the file with bash seems to fix the issue, so I think there's something strange going on with fread.

# Output of sessionInfo()

R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3.10.3
LAPACK: /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3.10.3

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] forcats_0.5.1     stringr_1.4.0     purrr_0.3.4       readr_2.1.2      
 [5] tidyr_1.2.0       tibble_3.1.3      ggplot2_3.3.5     tidyverse_1.3.1  
 [9] data.table_1.14.2 dplyr_1.0.7      

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7       cellranger_1.1.0 pillar_1.6.2     compiler_4.1.0  
 [5] dbplyr_2.1.1     tools_4.1.0      jsonlite_1.7.2   lubridate_1.7.10
 [9] lifecycle_1.0.0  gtable_0.3.0     pkgconfig_2.0.3  rlang_0.4.11    
[13] reprex_2.0.1     cli_3.0.1        rstudioapi_0.13  DBI_1.1.1       
[17] haven_2.4.3      xml2_1.3.2       withr_2.4.2      httr_1.4.2      
[21] fs_1.5.0         generics_0.1.0   vctrs_0.3.8      hms_1.1.0       
[25] grid_4.1.0       tidyselect_1.1.1 glue_1.6.2       R6_2.5.1        
[29] fansi_0.5.0      readxl_1.3.1     tzdb_0.1.2       modelr_0.1.8    
[33] magrittr_2.0.1   backports_1.2.1  scales_1.1.1     ellipsis_0.3.2  
[37] rvest_1.0.1      assertthat_0.2.1 colorspace_2.0-2 utf8_1.2.2      
[41] stringi_1.7.3    munsell_0.5.0    broom_0.7.9      crayon_1.4.1