Mysqlnd: packet headers bigger than buffer (4096) miss DB errors and put driver in inconsistent state

Description

The following was observed when using MySQL to communicate with Vitess, a MySQL database sharding layer. It is much more verbose in its errors (it prints the query back), so the following is triggered. I realize it's with Vitess, but mysqlnd shouldn't rely on conciseness of errors to keep from failing.

The situation:

The mysqlnd driver reserves 4096 bytes for the header buffer and when it's too big, it gives an error only visible in debug tracing. See here.

You then get a PHP warning "Error reading result set's header". This is a warning, not an exception, even though PDO is set to exceptions.

Because it didn't finish parsing the packet, it now doesn't realize this query failed. Vitess rejected a NULL in a NOT NULL column. All php did was print a warning about the header, and then next queries complain about active queries in unbuffered mode (which is required for Vitess).

The following code triggers it but only on Vitess:

create table mychars(val text not null);

<?php

// Won't fail on stock MySQL, but will on Vitess

function connect_to_database_base($host, $user, $password, $dbname) : PDO {
  $db = new PDO('mysql:host=' . $host . ';dbname=' . $dbname, $user, $password, array(
    PDO::ATTR_ERRMODE              =>  PDO::ERRMODE_EXCEPTION,
    PDO::MYSQL_ATTR_INIT_COMMAND   =>  'SET NAMES utf8 COLLATE utf8_unicode_ci',
    PDO::MYSQL_ATTR_USE_BUFFERED_QUERY => false
  ));
  return $db;
}

function connect_to_database() : PDO {
  return connect_to_database_base("172.31.4.54", "user_dev", "password", "dbname");
  
}

$db = connect_to_database();

// Create a long query, with invalid data.
$myquery = "INSERT INTO mychars(val) VALUES ('first'), ('bla ";
for ($i = 0; $i < 800; $i++) {
  $myquery .= " bla ";
}
$myquery .= "'), (NULL)";

print("I'm going to perform this query:\n\n$myquery\n\n");

$db->exec($myquery);

print("\nAs you can see, it generated an uncatchable 'warning'. PDO will now not see that it was an error.\n");

print("The only reason this error is detected, because we can't keep using the unbuffered connection. Doing a select:\n\n");
$db->query('SELECT * FROM mychars LIMIT 1');

print("End\n");
?>

Resulted in this output:

I'm going to perform this query:

INSERT INTO mychars(val) VALUES ('first'), ('bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla  bla '), (NULL)


Warning: PDO::exec(): Error reading result set's header in /root/reproduce_truncated_result_header_longtext.php on line 27

As you can see, it generated an uncatchable 'warning'. PDO will now not see that it was an error.
The only reason this error is detected, because we can't keep using the unbuffered connection. Doing a select:


Fatal error: Uncaught PDOException: SQLSTATE[HY000]: General error: 2014 Cannot execute queries while other unbuffered queries are active.  Consider using PDOStatement::fetchAll().  Alternatively, if your code is only ever going to run against mysql, you may enable query buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute. in /root/reproduce_truncated_result_header_longtext.php:32
Stack trace:
#0 /root/reproduce_truncated_result_header_longtext.php(32): PDO->query('SELECT * FROM m...')
#1 {main}
  thrown in /root/reproduce_truncated_result_header_longtext.php on line 32

But I expected this instead:

I'm going to perform this query:

INSERT INTO mychars(val) VALUES ('first'), ('bla  bla  bla '), (NULL)


Fatal error: Uncaught PDOException: SQLSTATE[23000]: Integrity constraint violation: 1048 target: vitess_production_clone.0.primary: vttablet: rpc error: code = InvalidArgument desc = Column 'val' cannot be null (errno 1048) (sqlstate 23000) (CallerID: userData1): Sql: "insert into mychars(val) values (:vtg1), (:vtg2), (null)", BindVars: {vtg1: "type:VARCHAR value:\"first\""vtg2: "type:VARCHAR value:\"bla  bla  bla \""} in /root/reproduce_truncated_result_header_longtext.php:27
Stack trace:
#0 /root/reproduce_truncated_result_header_longtext.php(27): PDO->exec('INSERT INTO myc...')
#1 {main}
  thrown in /root/reproduce_truncated_result_header_longtext.php on line 27

• I expect that the fact that the query gave an error was properly registered (it is not now)
• I also expect that the connection doesn't keep a cursor open and so doesn't fail on subsequent queries.

Tested with 8.1.11 tarball, compiled with debugging.

PHP Version

PHP 8.1.11, compiled debug version

Operating System

Ubuntu 22.04

ext/mysqlnd/mysqlnd_wireprotocol.c の、リンク先のパケットヘッダー処理の周辺から始めます。提供されている PDO の非バッファクエリの例で長いエラーのケースを再現し、その後、ヘッダーエラーがエラー報告とカーソル状態にどのような影響を与えるかを追跡します。データベースエラーが期待どおりに表面化し、非バッファクエリがアクティブなままになっていることが原因で後続のクエリが失敗しなければ完了です。