BurntSushi / BurntSushi/rust-csv

Request: Reader::clear_headers()

Open
#164 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
2k
Forks
257
PR merge metrics
No merged PRs in 30d

Description

Version: 1.1.1

At work I have a task to parse a CSV file containing two tables one after another with different field names and cardinality (I don't have any control over this format):

```csv
# date the file was generated, only ever one row
year,month,day,hour.minute,second
2019,07,19,07,58,17
# proprietary data, arbitrary number of rows
foo,bar,baz,quxx
0,0,0,0
1,1,1,1
# ... etc
```

I decided to parse it into two different structs using Serde and combine them manually:

```rust
#[derive(Debug, Deserialize, PartialEq, Eq)]
pub struct FileDate {
pub year: u16,
pub month: u8,
pub day: u8,
pub hour: u8,
pub minute: u8,
pub second: u8,
}

#[derive(Debug, Deserialize, PartialEq, Eq)]
pub struct ProprietaryData {
pub foo: u64,
pub bar: u64,
pub baz: u64,
pub quux: u64,
}

pub struct DataWithDate {
date: FileDate,
data: ProprietaryData,
}
```

My first instinct was to use two separate readers:

```rust
pub fn parse>(path: P) -> Result {
let mut file = File::open(path)?;

let date =ReaderBuilder::new()
.from_reader(&mut file)
.into_deserialize::()
.next()
.ok_or_else(|| failure::err_msg("did not find FileDate in balance CSV"))??;

// then read the rest of the data
let data = ReaderBuilder::new()
.from_reader(&mut file)
.into_deserialize()
.collect::, csv::Error>>()?;

Ok(DataWithDate { date, data })
}
```

However, I found that didn't work because `Reader` uses `BufReader` internally and so the first one is throwing away required data when it's dropped.

I then figured I could use the same reader and replace the headers, but it required setting `.flexible(true)` which weakens validation:

```rust
pub fn parse>(path: P) -> Result {
let mut reader = ReaderBuilder::new()
.flexible(true)
.from_path(path)?;

let date = reader
.deserialize::()
.next()
.ok_or_else(|| failure::err_msg("did not find FileDate in balance CSV"))??;

// read the headers for the data and set it
let mut data_header = StringRecord::new();

// this requires setting `.flexible(true)` otherwise it will return an error about the cardinality of the headers row
if !reader.read_record(&mut data_header)? {
return Err(failure::err_msg("failed to read data header"));
}

reader.set_headers(data_header);

// then read the rest of the data
let data = reader
.into_deserialize()
.collect::, csv::Error>>()?;

Ok(DataWithDate { date, data })
}
```

I would like a `Reader::clear_headers()` method that makes it behave as if it was at the beginning of a new table:

```rust
pub fn parse>(path: P) -> Result {
let mut reader = ReaderBuilder::new()
.flexible(true)
.from_path(path)?;

let date = reader
.deserialize::()
.next()
.ok_or_else(|| failure::err_msg("did not find FileDate in balance CSV"))??;

// clear the headers so it reads the new ones
reader.clear_headers();

// then read the rest of the data
let data = reader
.into_deserialize()
.collect::, csv::Error>>()?;

Ok(DataWithDate { date, data })
}
```

I don't really like having to set `.flexible(true)` because it's only necessary for reading that second headers row; I'd prefer to be more robust against format changes and mistakes (which is the same reason I don't want to set `.has_headers(false)`).

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.