BurntSushi / BurntSushi/rust-csv
Request: Reader::clear_headers()
- Dominant language
- Rust
- Stars
- 2k
- Forks
- 257
- PR merge metrics
- No merged PRs in 30d
Description
Version: 1.1.1
At work I have a task to parse a CSV file containing two tables one after another with different field names and cardinality (I don't have any control over this format):
```csv
# date the file was generated, only ever one row
year,month,day,hour.minute,second
2019,07,19,07,58,17
# proprietary data, arbitrary number of rows
foo,bar,baz,quxx
0,0,0,0
1,1,1,1
# ... etc
```
I decided to parse it into two different structs using Serde and combine them manually:
```rust
#[derive(Debug, Deserialize, PartialEq, Eq)]
pub struct FileDate {
pub year: u16,
pub month: u8,
pub day: u8,
pub hour: u8,
pub minute: u8,
pub second: u8,
}
#[derive(Debug, Deserialize, PartialEq, Eq)]
pub struct ProprietaryData {
pub foo: u64,
pub bar: u64,
pub baz: u64,
pub quux: u64,
}
pub struct DataWithDate {
date: FileDate,
data: ProprietaryData,
}
```
My first instinct was to use two separate readers:
```rust
pub fn parse>(path: P) -> Result {
let mut file = File::open(path)?;
let date =ReaderBuilder::new()
.from_reader(&mut file)
.into_deserialize::()
.next()
.ok_or_else(|| failure::err_msg("did not find FileDate in balance CSV"))??;
// then read the rest of the data
let data = ReaderBuilder::new()
.from_reader(&mut file)
.into_deserialize()
.collect::, csv::Error>>()?;
Ok(DataWithDate { date, data })
}
```
However, I found that didn't work because `Reader` uses `BufReader` internally and so the first one is throwing away required data when it's dropped.
I then figured I could use the same reader and replace the headers, but it required setting `.flexible(true)` which weakens validation:
```rust
pub fn parse>(path: P) -> Result {
let mut reader = ReaderBuilder::new()
.flexible(true)
.from_path(path)?;
let date = reader
.deserialize::()
.next()
.ok_or_else(|| failure::err_msg("did not find FileDate in balance CSV"))??;
// read the headers for the data and set it
let mut data_header = StringRecord::new();
// this requires setting `.flexible(true)` otherwise it will return an error about the cardinality of the headers row
if !reader.read_record(&mut data_header)? {
return Err(failure::err_msg("failed to read data header"));
}
reader.set_headers(data_header);
// then read the rest of the data
let data = reader
.into_deserialize()
.collect::, csv::Error>>()?;
Ok(DataWithDate { date, data })
}
```
I would like a `Reader::clear_headers()` method that makes it behave as if it was at the beginning of a new table:
```rust
pub fn parse>(path: P) -> Result {
let mut reader = ReaderBuilder::new()
.flexible(true)
.from_path(path)?;
let date = reader
.deserialize::()
.next()
.ok_or_else(|| failure::err_msg("did not find FileDate in balance CSV"))??;
// clear the headers so it reads the new ones
reader.clear_headers();
// then read the rest of the data
let data = reader
.into_deserialize()
.collect::, csv::Error>>()?;
Ok(DataWithDate { date, data })
}
```
I don't really like having to set `.flexible(true)` because it's only necessary for reading that second headers row; I'd prefer to be more robust against format changes and mistakes (which is the same reason I don't want to set `.has_headers(false)`).
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.