maennchen / maennchen/ZipStream-PHP

Make CHUNKED_READ_BLOCK_SIZE configurable

Open
#436 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
PHP
Stars
1.9k
Forks
112
Avg merge
14m
Merged PRs (30d)
4

Description

### Description

Hi,

While benchmarking large XLSX generation with ZipStream, I noticed that the current fixed `CHUNKED_READ_BLOCK_SIZE` of 16 MiB has a surprisingly large impact on peak PHP memory usage.

I tested the same 100k × 10 XLSX workload (~1 million cells) with different values for `CHUNKED_READ_BLOCK_SIZE` with `defaultEnableZeroHeader=false` for compatibility.

It's really this part of the code that I'd like to make configurable

```php
...
$readLength = min(
($this->maxSize ?? PHP_INT_MAX) - $this->uncompressedSize,
($this->exactSize ?? PHP_INT_MAX) - $this->uncompressedSize,
self::CHUNKED_READ_BLOCK_SIZE
);

$data = fread($stream, $readLength);
...
```

### Results

#### Windows

| Chunk size | Time | Peak memory |
| ------------------------ | -----: | ----------: |
| 16 MiB (current default) | 1.88 s | 36.7 MB |
| 4 MiB | 1.91 s | 12.7 MB |
| 1 MiB | 1.85 s | 5.71 MB |
| 256 KiB | 1.85 s | 5.71 MB |

#### Linux / WSL, PHP 8.4.8

| Chunk size | Time | Peak memory |
| ------------------------ | -----: | ----------: |
| 16 MiB (current default) | 1.32 s | 32.9 MB |
| 4 MiB | 1.32 s | 8.8 MB |
| 1 MiB | 1.30 s | 2.95 MB |
| 256 KiB | 1.31 s | 2.95 MB |

The result is quite striking: reducing the read chunk from 16 MiB to 1 MiB lowers peak memory by roughly **6–11×**, with no measurable throughput regression.

Going below 1 MiB did not reduce peak memory further in this workload, so 1 MiB seems to be a good practical sweet spot.

### Proposal

Would you consider making the chunk size configurable, for example through a constructor option?

Something like:

```php
new ZipStream(
// ...
readChunkSize: 1024 * 1024,
);
```

This would allow applications that care about bounded memory usage to choose a smaller chunk without patching ZipStream internals.

It may also be worth reconsidering the 16 MiB default itself. Based on these measurements, 1 MiB appears to provide essentially identical performance while substantially reducing peak memory. However, simply making it configurable would already solve the use case without requiring a default change.

If useful, I can also provide a small standalone benchmark reproducer.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the ZipStream constructor and the code using CHUNKED_READ_BLOCK_SIZE, especially the fread call shown in the issue. Trace how constructor options are stored and used, then inspect the existing test structure before adding coverage. Done means callers can set readChunkSize while the current behavior remains the default.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.