php / php/php-src

unicode string corruption when using PDO_ODBC with FreeTDS 1.3.6 and MS SQL Server 2016

Open
#9,498 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug Extension: pdo_odbc Status: Needs Triage
Dominant language
C
Stars
40.4k
Forks
8.1k
Avg merge
2d 13h
Merged PRs (30d)
96

Description

Description

The following code:

<?php
// setup database connection
$u = getenv('DB_USERNAME');
$p = getenv('DB_PASSWORD');
$dsn = 'odbc:testUTF8';
$db = new \PDO($dsn, $u, $p);
$db->setAttribute(\PDO::ATTR_ERRMODE, \PDO::ERRMODE_EXCEPTION);

$constant_tests = [
    "SELECT CAST(0x72006f017e016f007600fd00 AS nvarchar(10)) AS pink_in_czech_binary_encoded_utf16;",
    "SELECT CAST(0x6b26 AS nvarchar(3)) AS Beamed_Eighth_Notes_binary_encoded_utf16;",
    "SELECT NCHAR(0x266b) AS Beamed_Eighth_Notes_nchar;",
    "SELECT N'♫' AS Beamed_Eighth_Notes_unicode_string_constant;",
    "SELECT N'růžový' AS pink_in_czech_unicode_string_constant;",
];
foreach($constant_tests as $t) {
    fwrite(STDERR, "Running query '$t'...\n");
    var_dump($db->query($t, \PDO::FETCH_ASSOC)->fetchAll());
}

Resulted in this output:

Running query 'SELECT CAST(0x72006f017e016f007600fd00 AS nvarchar(10)) AS pink_in_czech_binary_encoded_utf16;'...
array(1) {
  [0]=>
  array(1) {
    ["pink_in_czech_binary_encoded_utf16"]=>
    string(9) "růžový"
  }
}
Running query 'SELECT CAST(0x6b26 AS nvarchar(3)) AS Beamed_Eighth_Notes_binary_encoded_utf16;'...
array(1) {
  [0]=>
  array(1) {
    ["Beamed_Eighth_Notes_binary_encoded_utf16"]=>
    string(3) "♫"
  }
}
Running query 'SELECT NCHAR(0x266b) AS Beamed_Eighth_Notes_nchar;'...
array(1) {
  [0]=>
  array(1) {
    ["Beamed_Eighth_Notes_nchar"]=>
    string(3) "▒▒"
  }
}
Running query 'SELECT N'♫' AS Beamed_Eighth_Notes_unicode_string_constant;'...
array(1) {
  [0]=>
  array(1) {
    ["Beamed_Eighth_Notes_unicode_string_constant"]=>
    string(3) "▒▒"
  }
}
Running query 'SELECT N'růžový' AS pink_in_czech_unicode_string_constant;'...
array(1) {
  [0]=>
  array(1) {
    ["pink_in_czech_unicode_string_constant"]=>
    string(9) "růžo"
  }
}

But I expected this output instead:

Running query 'SELECT CAST(0x72006f017e016f007600fd00 AS nvarchar(10)) AS pink_in_czech_binary_encoded_utf16;'...
array(1) {
  [0]=>
  array(1) {
    ["pink_in_czech_binary_encoded_utf16"]=>
    string(9) "růžový"
  }
}
Running query 'SELECT CAST(0x6b26 AS nvarchar(3)) AS Beamed_Eighth_Notes_binary_encoded_utf16;'...
array(1) {
  [0]=>
  array(1) {
    ["Beamed_Eighth_Notes_binary_encoded_utf16"]=>
    string(3) "♫"
  }
}
Running query 'SELECT NCHAR(0x266b) AS Beamed_Eighth_Notes_nchar;'...
array(1) {
  [0]=>
  array(1) {
    ["Beamed_Eighth_Notes_nchar"]=>
    string(3) "♫"
  }
}
Running query 'SELECT N'♫' AS Beamed_Eighth_Notes_unicode_string_constant;'...
array(1) {
  [0]=>
  array(1) {
    ["Beamed_Eighth_Notes_unicode_string_constant"]=>
    string(3) "♫"
  }
}
Running query 'SELECT N'růžový' AS pink_in_czech_unicode_string_constant;'...
array(1) {
  [0]=>
  array(1) {
    ["pink_in_czech_unicode_string_constant"]=>
    string(9) "růžový"
  }
}
PHP Version

PHP 8.1.7

Operating System

Debian GNU/Linux testing/bookworm/unstable

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the provided PHP script, PDO::query(), and fetchAll() through PDO_ODBC with FreeTDS 1.3.6 and SQL Server 2016. Compare the returned values for NCHAR and N-prefixed string constants with the expected Unicode strings; done means all listed queries return the expected characters without corruption.

Written by the indexing model from the issue text.

Assessment

Tech stack
php, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.