Increase FILLUNIT (currently 5kB) in main/rfc1867.c otherwise large uploads are inefficients and slow

Description

I recently stumbled upon a very simple file upload code which was quite slow. It turned out that it was IO-limited while writing the temporary file to the disk. A strace showed that PHP would spoonfeed the incoming network data to the disk :

16:51:04 read(6, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 4142) = 4142
16:51:04 read(6, "\1\5\0\1 \0\0\0", 8)  = 8
16:51:04 read(6, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 977) = 977
16:51:04 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5119) = 5119
16:51:04 read(6, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5119) = 5119
16:51:04 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5119) = 5119
16:51:04 read(6, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 2096) = 2096
16:51:04 read(6, "\1\5\0\1 \0\0\0", 8)  = 8
16:51:04 read(6, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 3023) = 3023
16:51:04 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 5119) = 5119

On the network/read side this is not very efficient, it uses a lot of syscalls (especially since there is a recurring 8-byte read() call). On my experiment (a n1-standard-8 GCE VM), a single process would use 50% CPU (on the system side, not the user one) with an upload proceeding at 15 MB/s.

On the disk/write side this is also inefficient, and runs quickly in real cloud limits : in my case a 100G "SSD" disk which GCE advertises as 48 MB/s and 3000 IO/s max; my benchmarks with fio confirmed those figures.

The problem with such a small FILLUNIT=5kB buffer, is that you can't get a better throughput than 3000*5 kB/s =15 MB/s. So you don't get the max througput of your disk, you top on IOPS first.

Of course that is amplified by the fact that is a VM and uses - as recommended by Google - the 'noop' IO scheduler and there is no write coalescing. I guess an IO scheduler could mitigate this on a bare-metal local storage. But I think running PHP on such a cloud VM (8 vCPUs/ 32GB RAM/ 100G "SSD") is very common. And the performance drop (and global pressure on the server storage) is very noticeable.

According to https://github.com/php/php-src/commit/44b68122c2ba0cf07cd837af160b3bd01527d081 FILLUNIT has been 5 kB for 23 years.

I would suggest it is time to bump it up. I would suggest at least 64 kB, GCE suggests 256 kB (https://cloud.google.com/compute/docs/disks/optimizing-pd-performance). Only one instance of this buffer is used at once, during the file upload loop (thus per-process), and can be released as soon as the loop is done.

Note that the value is used to allocate a second transient buffer on the stack in multipart_buffer_read_body() - and that might abuse the stack (which defaults to 1MB on Windows and 8MB on Linux if I'm not mistaken).

PHP Version

PHP 8.1.27

Operating System

Debian 10

Start in main/rfc1867.c by locating FILLUNIT and multipart_buffer_read_body(), then review how the buffer sizes are allocated during uploads. Use the reported strace and fio measurements as performance context, and account for the transient stack buffer. Done means an agreed larger value improves upload throughput without compromising upload behavior or stack usage.