URL table function does not heed input_format_max_block_wait_ms

Start with ReadWriteBufferFromHTTP and ReadBuffer::poll(), then trace how IRowInputFormat::read() and FormatFactory::getInputImpl() handle input_format_max_block_wait_ms for URL reads. Reproduce the behavior with a bursty URL producer and verify that the setting affects waiting without introducing the reported configuration error.

Company or project name

eventus

Describe the unexpected behaviour

The input_format_max_block_wait_ms does not seem to work with the URL table function. I have tested in version 26.3. Here's what the AI has to say about my findings...

==================================================================================
streaming-inserts work (PR #94509 (https://github.com/ClickHouse/ClickHouse/pull/94509)). The mechanism lives in IRowInputFormat::read():

if (max_block_wait_ms != 0 && num_rows > 0)
{
UInt64 elapsed_ms = watch.elapsedMilliseconds();
if (elapsed_ms >= max_block_wait_ms)
break;
UInt64 remaining_us = (max_block_wait_ms - elapsed_ms) * 1000;
if (!getReadBuffer().poll(remaining_us)) // <-- the load-bearing call
break;
}

So the timeout cut depends on ReadBuffer::poll() reporting readiness. The base implementation is:

virtual bool poll(size_t /* timeout_microseconds */) { return true; }

Why url() misses out. ReadBufferFromPocoSocketBase — the server reading an INSERT off the client connection — overrides poll(). ReadWriteBufferFromHTTP, which is what url() reads through, does not; it overrides nextImpl, seek, readBigAt, setReadUntilPosition, etc., but no poll. So for url(), poll() returns true instantly, the parser goes straight into reading the next row, and that call blocks on the socket until bytes arrive. The timeout can never interrupt a wait — it can only trim a block between rows that are already arriving, which is exactly the case where you didn't need it.

Two more gates:

• FormatFactory::getInputImpl() throws BAD_ARGUMENTS if input_format_max_block_wait_ms > 0 without input_format_connection_handling. Setting it on a SELECT ... FROM url(...) alone is an error, not a silent ignore.
• connection_handling forces parallel_parsing = false. For a url() read that's a straight pessimization with no compensating benefit.

And it's IRowInputFormat only — url(..., Parquet), Native, ORC etc. never touch this code path regardless.
Sources:

• Format settings — input_format_max_block_wait_ms (https://clickhouse.com/docs/reference/settings/formats/input-format)
• ClickHouse 26.2 release blog — streaming inserts (https://clickhouse.com/blog/clickhouse-release-26-02)
• PR #94509 — Streaming inserts (https://github.com/ClickHouse/ClickHouse/pull/94509)

How to reproduce

Use the URL table function with a producer that is bursty.

Expected behavior

Have input_format_max_block_wait_ms affect the URL table function.

Error message and/or stacktrace

No response

Additional context

No response