Repetitive quotes ("") expanding, in the end causing crash

Open
#3,509 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
r
Domain
data

Research direction

Start with the minimal reproducible example using fread() and fwrite() on testDT.txt and testDT.csv, then repeat the read/write cycle while inspecting the quoted values. Compare the behavior across the CRAN and development versions; done means repetitive quotes no longer grow between cycles and fwrite() no longer crashes.

Written by the indexing model from the issue text.

Description

fread fwrite

THANK YOU FOR ALL YOUR GREAT WORK! You enable my research efforts to be efficient on a daily basis.
I have searched: news, issues, milestones, stackoverflow, all to no avail.
What happens is, somehow one of my variables accumulates an ever-growing string of ""'s after being read and re-written by fread. This goes on, until the value assumes a size that causes fwrite to crash:
fwrite(DT , "testDT.csv")

*** caught segfault ***
address 0x7f9ce33ff000, cause 'memory not mapped'

Traceback:
1: fwrite(DT, "testDT.csv")

I have worked around this problem by deleting the many repetitive ""'s using gsub, but I don't know why they arise in the first place. I suspect there is something in the way fread and fwrite handles ""'s (I am using R and data.table to sort references,(I admit this is not a very frequent use case) and my reference programs may be putting in some of my authors have unusual letters in their names).
I have tested this both in the data.table version from CRAN and the development version, installed as described on your home page.
Thank you for your time!

# [Minimal reproducible example]
library(data.table)
DT <- fread("testDT.txt") ##Using .txt as GitHub does not like .csv
testDT.txt
fwrite("DT" , "testDT.csv")
DT <- fread("testDT.csv")
DT
##You will notice that the number of ""'s has grown now - this increases by each cycle.

# Output of sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.2 LTS

Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_DK.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_DK.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_DK.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_DK.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] data.table_1.12.3

loaded via a namespace (and not attached):
[1] compiler_3.5.3 tools_3.5.3

Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.