microsoft / microsoft/DacFx

Corrupt data in JSON type column after Importing database from .bacpac in Azure SQL Database server

Open
#816 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
460
Forks
29
Avg merge
4d 9h
Merged PRs (30d)
7

Description

- SqlPackage or DacFx Version:
- .NET Framework (Windows-only) or .NET Core:

I've done an **Import database** (action in the top menu in Overview blade) in an Azure SQL Database server,

- chosen a previously created .bacpac file located in Azure Storage Account as a source -> this was created from a database on the same server

- checked that the target collation matches with the collation chosen in the original source database

- selected the same account for the target db

After restore finished, I started data validation check and found following

- data in **native** **JSON type columns** were **partially** **corrupted** (see below for details)

- rest of the data seem to be correct

**Corrupt Data** (Description + Analysis + Examples)
Corrupt data is located in text values of our JSON properties, in place where special german characters were located, e.g. ü, ä, ö. It seems the characters were read back into the new database, the original UTF-8 bytes were misinterpreted as Latin-1 and stored as such. E.g.

Original JSON fragment (correct)

```javascript
{
"CSName": "_Gezählt",
"Text": "Gezählt",
"Note": "Noun, singular. Located in label texts on some screens.",
"Elements": ["scrCountLists.Label2.Text", "scrKonso.lbStatus_1.Text"]
}
```

Corrupted JSON fragment (see values in CSName and Text)

```javascript
{
"CSName": "_Gez\u00C3\u00A4hlt",
"Text": "Gez\u00C3\u00A4hlt",
"Note": "Noun, singular. Located in label texts on some screens.",
"Elements": [
"scrCountLists.Label2.Text",
"scrKonso.lbStatus_1.Text"
]
}
```

Affected table definition
CREATE TABLE [dbo].[Translation]

(

[Locale] NVARCHAR(10) NOT NULL PRIMARY KEY

, [Definition] JSON NOT NULL

)

Corrupt data are in the [Definition] column.

Service: Azure SQL Database

I can provide more examples upon request.

Thank you for fixing this, Microsoft.

Kind regards,
Jozef Sčensný
SWE | DBA | Data Engineer

- Environment (local platform and source/target platforms):

**Steps to Reproduce:**

1.
2.

**Did this occur in prior versions? If not - which version(s) did it work in?**

(DacFx/SqlPackage/SSMS/Azure Data Studio)

Contributor guide

Open the contributing guide

Research direction

Start with the Azure SQL Database Overview blade's Import database action and reproduce the report using the supplied dbo.Translation table and JSON Definition column. Compare the source and restored values for German characters, recording the SqlPackage or DacFx and platform versions; done means the imported JSON preserves the original text.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, sql
Domain
cloud, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.