The Parquet read holds two copies of the table, and that is where the peak is

Split out of #405, which measured the peak and found it is reached during the Parquet read rather than during query execution. The numbers are in the comment there. Reading sf1 lineitem, 392 MB and 6,001,215 rows, firepanda holds 1.157 GB and peaks at 3.415 GB, where duckdb holds 0.977 GB and peaks at 1.492 GB and polars holds 0.854 GB and peaks at 1.681 GB.

Two problems, and they are independent.

## The transient

`Session.run` in `firepanda/io/parquet.mojo` asks DuckDB for a materialised result, fetches every chunk out of it into a list, and only then calls `assemble` to copy the lot into firepanda arrays. For the length of that copy DuckDB's whole result and firepanda's whole frame are both resident. That is 0.98 plus 1.16 GB on lineitem before DuckDB's own read buffers go on top, and it accounts for most of the 3.4 GB.

Streaming the result is the obvious answer and it is already ruled out. The note in `_execute` records 400 ms against 270 ms on ten million rows, because a streaming result is produced by one thread pulling on the pipeline while a materialised one is produced by every thread DuckDB has. Whatever we do here has to keep the materialised execution.

The blocker is that `assemble` cannot allocate until it has seen every batch. It sizes each column in a first parallel pass, because a string column's payload size is the sum over the batches and is not implied by the row count, and only then allocates the sinks and fills them. So there is no way to pre-allocate the frame from `duckdb_row_count` alone and copy chunk by chunk.

The route that looks right is to read in groups rather than all at once. Fetch a fixed number of chunks, assemble that group, destroy those chunks, and keep the group as one chunk of a `ChunkedArray`. Peak then holds the finished frame plus one group instead of the finished frame plus the whole result. The cost is that a Parquet read stops producing single chunk columns, which it does today, so this needs the query side measured before it lands and not after.

- [x] Measure where the peak actually goes through the read, staged numbers in the comment below: 1241 MB for DuckDB's materialised result, 1290 MB for fetching the chunks out of it, 682 MB to describe them as Arrow, 1118 MB to assemble the frame
- [ ] If the cost is acceptable, read in groups and release each group's chunks once it is assembled, which recovers the 1290 MB fetch copy and not the 1241 MB result
- [ ] If it is not, size the columns from a metadata pass instead and pre-allocate, which needs DuckDB to tell us the string payload sizes up front
- [ ] Re-measure peak against duckdb and polars on the same file

## The resident size

We hold 1.157 GB where polars holds 0.854 GB for the same rows, thirty five per cent more. Not looked into yet. `_direct_layout` claims every column is nullable, which costs a validity bitmap on columns that have no nulls, but that is only twelve megabytes across the sixteen columns and does not explain it.

- [x] Account for the gap column by column, done in the comment below: all of it is the sixteen byte string view against Arrow's four byte offset
- [ ] Dictionary encode a string column DuckDB read from a Parquet dictionary, or one whose distinct count is small enough to pay, worth 337 MB on lineitem
- [ ] Drop the validity bitmap on a column DuckDB reports no nulls for

Start in firepanda/io/parquet.mojo, reading Session.run, _execute, and assemble, then review the measurements in #405 and this issue. Measure grouped chunk assembly against the current materialised read before choosing an approach. Done means peak memory is re-measured against DuckDB and Polars, with the resident-size tasks addressed if applicable.

Written by the indexing model from the issue text.