maennchen / maennchen/ZipStream-PHP
Make CHUNKED_READ_BLOCK_SIZE configurable
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 1.9k
- Forks
- 112
- Avg merge
- 14m
- Merged PRs (30d)
- 4
Description
### Description
Hi,
While benchmarking large XLSX generation with ZipStream, I noticed that the current fixed `CHUNKED_READ_BLOCK_SIZE` of 16 MiB has a surprisingly large impact on peak PHP memory usage.
I tested the same 100k × 10 XLSX workload (~1 million cells) with different values for `CHUNKED_READ_BLOCK_SIZE` with `defaultEnableZeroHeader=false` for compatibility.
It's really this part of the code that I'd like to make configurable
```php
...
$readLength = min(
($this->maxSize ?? PHP_INT_MAX) - $this->uncompressedSize,
($this->exactSize ?? PHP_INT_MAX) - $this->uncompressedSize,
self::CHUNKED_READ_BLOCK_SIZE
);
$data = fread($stream, $readLength);
...
```
### Results
#### Windows
| Chunk size | Time | Peak memory |
| ------------------------ | -----: | ----------: |
| 16 MiB (current default) | 1.88 s | 36.7 MB |
| 4 MiB | 1.91 s | 12.7 MB |
| 1 MiB | 1.85 s | 5.71 MB |
| 256 KiB | 1.85 s | 5.71 MB |
#### Linux / WSL, PHP 8.4.8
| Chunk size | Time | Peak memory |
| ------------------------ | -----: | ----------: |
| 16 MiB (current default) | 1.32 s | 32.9 MB |
| 4 MiB | 1.32 s | 8.8 MB |
| 1 MiB | 1.30 s | 2.95 MB |
| 256 KiB | 1.31 s | 2.95 MB |
The result is quite striking: reducing the read chunk from 16 MiB to 1 MiB lowers peak memory by roughly **6–11×**, with no measurable throughput regression.
Going below 1 MiB did not reduce peak memory further in this workload, so 1 MiB seems to be a good practical sweet spot.
### Proposal
Would you consider making the chunk size configurable, for example through a constructor option?
Something like:
```php
new ZipStream(
// ...
readChunkSize: 1024 * 1024,
);
```
This would allow applications that care about bounded memory usage to choose a smaller chunk without patching ZipStream internals.
It may also be worth reconsidering the 16 MiB default itself. Based on these measurements, 1 MiB appears to provide essentially identical performance while substantially reducing peak memory. However, simply making it configurable would already solve the use case without requiring a default change.
If useful, I can also provide a small standalone benchmark reproducer.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the ZipStream constructor and the code using CHUNKED_READ_BLOCK_SIZE, especially the fread call shown in the issue. Trace how constructor options are stored and used, then inspect the existing test structure before adding coverage. Done means callers can set readChunkSize while the current behavior remains the default.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100