[FEA] Modernize CSV reader and expand reader options

### Background

The CSV reader in cuDF/libcudf is a common IO interface for ingesting raw data, and is frequently the first IO interface that new users test when getting started with RAPIDS. There have been many improvements to the CSV reader over the years, but much of the implementation has remained the same from its introduction in #3213 and rework in #5024. We see several opportunities to address the [CSV reader continuous improvement](https://github.com/rapidsai/cudf/milestone/12) milestone, and this story associates open issues with particular functions and kernels in the CSV reading process.

### Step 1: Decompression and preprocessing
The CSV reader begins with host-side processing in `select_data_and_row_offsets`. With the exception of decompression, we would like to migrate this processing to be done device side and refactor this function to use a kvikIO data source. Note, this refactor could also include adding support for the `header` parameter and `byte_range` at the same time ([code pointer](https://github.com/rapidsai/cudf/blob/b798a70d608cbbe2c7f372a8c21354455ba56f74/cpp/src/io/csv/reader_impl.cu#L442)).

The initial processing interacts with several issues:
* #13797 is small story issue about this topic
* #4999 batch the full read as small chunks
* #5142
* #11728 describes how the initial byte range parsing to find the first row assumes the byte_range starts in an unquoted state. If a user provides a byte_range that starts in a quoted field, then the reader will fail! The solution described in this issue interacts the next step "identify row offsets".
* #12255 needs investigation
* #12582 return empty `metadata.schema_info` when column names are autogenerated

### Step 2: Identify row offsets (delimiters)
The next step is identifying record delimiters and computing row offsets in `load_data_and_gather_row_offsets` (invoked by `select_data_and_row_offsets`). This algorithm operates in three main steps: `gather_row_offsets` called with empty data, `select_row_context`, and `gather_row_offsets` called with row context data. The row context state machine is difficult to refactor because it uses a custom data representation that stores several logical values within a single 32-bit or 64-bit physical type ([code pointer](https://github.com/rapidsai/cudf/blob/b798a70d608cbbe2c7f372a8c21354455ba56f74/cpp/src/io/csv/csv_gpu.hpp#L64)). The row context tracks whether the content is in a comment block or in a quoted block.
* `gather_row_offsets` runs a [4-state](https://github.com/rapidsai/cudf/blob/b798a70d608cbbe2c7f372a8c21354455ba56f74/cpp/src/io/csv/csv_gpu.hpp#L40) "row context" state machine over 16 KB blocks of characters and returns the number of un-quoted, un-commented record delimiters from the block given each possible initial state
* `select_row_context` is invoked in a host-side loop over `row_ctr` data for each 16 KB block, starting from a `state 0` initial context.
* `gather_row_offsets` is called in a second pass with a valid `all_row_offsets` data parameter.

Major design topics:
* We should consider a larger refactor of the "identify row offsets" code based on using a new FST instance ([code pointer](https://github.com/rapidsai/cudf/tree/branch-23.10/cpp/src/io/fst)). Using an FST instance would easily allow us to add additional states beyond the existing 4-state machine. Please refer to the [ParPaRaw paper](https://arxiv.org/pdf/1905.13415.pdf) from Elias Stehle et al for more information about parallel algorithms for CSV parsing.
* To unblock Spark-RAPIDS usage of the CSV, we may also choose to support user-provided `all_row_offsets` parameter to the read function or as a reader option. This would allow Spark to bypass the first `gather_row_offsets` pass and `select_row_context` in `load_data_and_gather_row_offsets`. When calling `read_csv` on a strings column, Spark already has the row offsets.
* Also note that refactoring the interface to provide row offsets is relevant to #11728, where we would want to provide pre-computed offsets. For this issue we might prefer a new detail API rather than new parameters in the public API - more design work is needed.

The row offsets algorithm interacts with several open issues:
* #6572 complex preprocessing or changes to the row context state machine.
* (Spark blocker) #11984 Pandas and Spark don't have the same escaping conventions, and the row offset state machine doesn't have an escaped state. Needs confirmation - does this impact the row offsets step?
* (Spark blocker) #11948 to handle misplaced quotes. The issue shows a file getting truncated so fields with misplaced quotes seem to compromise the row offset data. #2398 suggests a workaround
* #6305 another quoting/escaping issue
* #13856 commented lines should not emit row offsets
* Issue n/a: Add unit tests for `gather_row_offsets` kernel

### Step 3: Determine column types
The next step is determining the data types for each column that does not map to a user-provided data type. The function `determine_column_types` completes this work by collecting the user-provided data types, and then calling `infer_column_types` to handle the unspecified data types. `infer_column_types` invokes the `detect_column_types`->`data_type_detection` kernel to collect statistics about the data in each field, and then use the conventions of the pandas CSV reader to select a column type.
* We should consider refactoring the "determine", "infer", and "detect" function names to improve clarity
* (good first issue) #14066 update thread indexing
* #5080 performance improvements for type inference
* (Spark blocker) #11984 `seek_field_end` supports escape characters within data fields. perhaps field traversal is already Spark-compatible
* (Spark blocker) #11948 misplaced quotes could fail with `seek_field_end`
* #6313 pandas doesn't infer as `float` if there are any nulls
* #9987 would change `seek_field_end`, maybe not much else
* Issue n/a: Add unit tests for `seek_field_end` kernel

### Step 4: Decode data and populate device buffers
The final step, `decode_data`, does another pass over the data to decode values according to the determined columns types. The kernel is `decode_row_column_data`->`convert_csv_to_cudf`
* (Spark blocker) #13892 trim white space . Probably a modest change to `trim_whitespaces_quotes`. Related to #6659
* (Spark blocker) #12145 add option to decode `""` as empty strings or `null`. Probably an additional parsing option.
* (Spark blocker) #11984 `convert_csv_to_cudf` also uses `seek_field_end` which nominally supports escape characters
* (Spark blocker) #11948 misplaced quotes could fail with `seek_field_end`
* #4001 support additional `nanValue` options
* #10599 float parsing consistency, this is probably a `wontfix`
* Use more efficient double-quote handling ([see link and TODO](https://github.com/rapidsai/cudf/blob/fe7412915a289e7a9469040ada1dcf74cda2c4d6/cpp/src/io/csv/reader_impl.cu#L868-L879))