Explore smashing datasets with arrow

### Context

We currently merge/smash datasets using pandas, but [Apache Arrow](https://arrow.apache.org/) may provide some speed/efficiency gains if their claims are true.

### Problem or idea

Reading and merging datasets, particularly for compendia, may be a bit of a bottleneck, and Apache Arrow claims great speed and efficiency, including the ability to efficiently pass data between R and python without copying. Whether that would play out for us is something that we should probably explore.

### Solution or next step

A small scale test/exploration period to be sure we can do all of the merging, filtering, and conversions that we need, and to benchmark Arrow-based functions in smashing.

I would expect the first steps would be to do some benchmarking of of reading our csv files using the `arrow` package (which can be done directly from s3, it seems) and joining them into large matrices.

The total conversion would be a pretty big change, I expect, so we'd definitely want to do some testing before committing to the effort!

Start with a small benchmark of reading the project's CSV files using the Python arrow package, including direct reads from S3 if feasible. Compare merging, filtering, conversions, and joining into large matrices against the current pandas approach; document whether the required operations work and the measured speed or efficiency gains before proposing a full conversion.

Written by the indexing model from the issue text.