optional arg that returns a list of parse symbols fread() used to intuit raw file

Aperta
#2,437 8 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Ferma
Stack tecnologico
r
Ambito
data

Direzione di ricerca

Start by reviewing fread() and the parser decisions described in this issue, then compare the proposed schema workflow with the prototype linked in datzen's scheme.md. Clarify which inferred delimiters and other parsing settings must be exposed, how the optional result should be represented, and what tests would demonstrate that the returned schema matches fread()'s behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

feature request fread

I think it would be a terrific feature addition if some kind of argument like fread(return_parse=TRUE) was implemented.

I know the developers have put in a lot of work to 'intuit / guess ' these type of delimiter characters in the fread() body, such as

https://github.com/Rdatatable/data.table/pull/2436
https://github.com/Rdatatable/data.table/issues/2248
https://github.com/Rdatatable/data.table/issues/2431

All of these internal parsing decisions would be a benefit if it was optionally returned to the end user outside of fread()

This request is motivated by the rant blogged about here
http://www.johnmyleswhite.com/notebook/2016/09/23/no-juice-for-you-csv-format-it-just-makes-you-more-awful/

Basically, i think a 'csv schema' would be useful for the type of workflow

raw csv -> fread -> user transforms data -> schema -> write external transformed data

I mocked up a protottype of a function that writes this basic schema here

https://github.com/mikejacktzen/datzen/blob/master/man_md/scheme.md

and realized that fread() internally tries to detect these types of raw parser characters

edit 10/24 for more use case context

I see this being useful if no pre-existing schema / preamble / yaml for csvy exists.
So the user wishes to create one painlessly using fread()

I do not think many people exhaustively lists the parser spec first.
If, like me, people ham handedly read the raw file into R with fread, we rely on fread to guess the spec (the hard work).
Once the fread generated spec is there, the user can then write out the scheme to whatever format they want, json, yaml, the preamble for the csvy.

Lingua principale
R
Stelle
3.9k
Fork
1.1k
Merge medio
14h 4m
PR unite (30g)
4

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Rdatatable/data.table

Tutte le issue di Rdatatable/data.table

Issue simili

Altre issue su R

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.