Writing large values

There was an interesting discussion on the Parquet mailing list about writing large values into Parquet files. Summary doc: https://docs.google.com/document/d/1nntcYM98PFSkHT70RexSBPtCnWqg1uRJ5_7m--ZgbsA/edit?tab=t.0#heading=h.k4r8orckhbx0

This did make me think about the situation in arrow-rs. The idea of offloading large completed pages to a temporary file and then reading them back when writing out sounds very interesting to me as a pragmatic solution to the problem.

Is this something we could / should support in arrow-rs?

Start with the linked Parquet mailing-list summary, then inspect arrow-rs's Parquet page-writing path and how completed pages are handled. Define whether large completed pages can be offloaded to a temporary file and read back during output, with support for writing large values as the completion criterion.

Written by the indexing model from the issue text.