BACPAC import fails (Msg 4002 / TDS stream ended unexpectedly) when a nullable JSON column is followed by another column and a row has NULL
- Dominant language
- C#
- Stars
- 460
- Forks
- 29
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 7
Description
- SqlPackage or DacFx Version: SqlPackage 170.3.93.6 (DacFxVersion 170.3.93.6, ModelSchemaVersion 2.9, DataStreamVersion 2.0.0.0)
- .NET Framework (Windows-only) or .NET Core: .NET Core 10.0.4 (sqlpackage as a dotnet global tool)
- Environment (local platform and source/target platforms): Local: Linux (Arch). Source and target: SQL Server 2025 RTM-CU4 (build 17.0.4035.5) on Linux/Ubuntu 24.04, Docker image mcr.microsoft.com/mssql/server:2025-CU4-ubuntu-24.04. Same instance for both source and target databases, Azure SQL Database is not required to reproduce
**Steps to Reproduce:**
1. Create a database with a single table containing a nullable JSON column followed by at least one other column, and insert one row whose JSON value is NULL:
```
CREATE DATABASE T;
GO
USE T;
GO
CREATE TABLE X (Json1 JSON NULL, Filler INT NOT NULL);
INSERT INTO X VALUES (NULL, 0);
GO
```
2. Export the database (succeeds, produces a ~3 KB BACPAC):
```
sqlpackage /Action:Export /SourceServerName:"" /SourceDatabaseName:T /SourceUser:SA /SourcePassword:"" /SourceTrustServerCertificate:True /TargetFile:T.bacpac
```
3. Import the BACPAC to a new database:
```
sqlpackage /Action:Import /SourceFile:T.bacpac /TargetServerName:"" /TargetDatabaseName:T2 /TargetUser:SA /TargetPassword:"" /TargetTrustServerCertificate:True
```
Expected: import succeeds.
Actual: fails on dbo.X with `Msg 4002: The incoming tabular data stream (TDS) protocol stream is incorrect. The stream ended unexpectedly.` thrown from `Microsoft.Data.SqlClient.SqlBulkCopy.CopyBatchesAsyncContinuedOnSuccess`. SqlPackage then enters an indefinite exponential-backoff retry loop (delays 0.25s, 0.5s, 1s, 2s, 4s, 8s, 16s, 32s, then capped at 120s) and continues retrying past retry 8.
The trigger is narrow and verified by isolation testing. The bug fires if and only if BOTH:
(a) the JSON-typed nullable column is followed by at least one other column in the table definition, AND
(b) at least one row has NULL in that JSON column.
Either condition alone passes. Both together fail deterministically, even on a single 1-row, 2-column table. The trailing column type does not matter (INT, JSON, etc. all trigger).
Possibly the same family as #8 (`_BIN2_UTF8` BACPAC fails on import with "Unexpected end of stream") and #765 (`sql_variant` + UTF-8 collation export failure).
**Did this occur in prior versions? If not - which version(s) did it work in?**
(DacFx/SqlPackage/SSMS/Azure Data Studio)
Tested only on SqlPackage 170.3.93.6, which is the current latest. I have not tested earlier SqlPackage versions. The first class JSON data type is new in SQL Server 2025, so any prior version comparison is constrained to SqlPackage versions released after JSON-type support was added to DacFx.
Contributor guide
Research direction
Reproduce the failure with the SQL schema and SqlPackage Export/Import commands in the issue, then trace the import failure from Microsoft.Data.SqlClient.SqlBulkCopy.CopyBatchesAsyncContinuedOnSuccess. Compare the nullable JSON-column case with the isolation cases described. Done means the BACPAC imports successfully without TDS errors or continued retrying.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, sql
- Domain
- databases, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100