[Future Work] Promote DirectMemmoveBuffer into Standard Bucket Path

ZonalFile manages concurrency through an in-house buffer that preallocates space, allowing workers to write directly to memory. It uses the ctypes.pythonAPI library to execute GIL-free writes, which avoids unnecessary data copies and prevents the global interpreter lock from creating a bottleneck. We call it DirectMemmoveBuffer as of today.

The standard path, by comparison, relies on b"".join. In this process, each worker generates its own byte object, and a new, combined copy is created when the join operation executes. This overhead limits throughput for standard buckets.

As for why we are addressing this now: we initially held off on integrating this with regional buckets due to concerns from the fsspec maintainers regarding the use of the ctypes API. If we are confident in the stability of our buffer, it is time to promote it to the standard bucket

Start by locating ZonalFile, DirectMemmoveBuffer, and the standard bucket write path. Compare the existing direct-memory implementation with the b"".join path, then review the ctypes.pythonAPI safety concerns before changing the shared path. Done means standard buckets use the buffer without regressions and relevant tests pass.