Corrupt data in JSON type column after Importing database from .bacpac in Azure SQL Database server
- Dominant language
- C#
- Stars
- 460
- Forks
- 29
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 7
Description
- SqlPackage or DacFx Version:
- .NET Framework (Windows-only) or .NET Core:
I've done an **Import database** (action in the top menu in Overview blade) in an Azure SQL Database server,
- chosen a previously created .bacpac file located in Azure Storage Account as a source -> this was created from a database on the same server
- checked that the target collation matches with the collation chosen in the original source database
- selected the same account for the target db
After restore finished, I started data validation check and found following
- data in **native** **JSON type columns** were **partially** **corrupted** (see below for details)
- rest of the data seem to be correct
**Corrupt Data** (Description + Analysis + Examples)
Corrupt data is located in text values of our JSON properties, in place where special german characters were located, e.g. ü, ä, ö. It seems the characters were read back into the new database, the original UTF-8 bytes were misinterpreted as Latin-1 and stored as such. E.g.
Original JSON fragment (correct)
```javascript
{
"CSName": "_Gezählt",
"Text": "Gezählt",
"Note": "Noun, singular. Located in label texts on some screens.",
"Elements": ["scrCountLists.Label2.Text", "scrKonso.lbStatus_1.Text"]
}
```
Corrupted JSON fragment (see values in CSName and Text)
```javascript
{
"CSName": "_Gez\u00C3\u00A4hlt",
"Text": "Gez\u00C3\u00A4hlt",
"Note": "Noun, singular. Located in label texts on some screens.",
"Elements": [
"scrCountLists.Label2.Text",
"scrKonso.lbStatus_1.Text"
]
}
```
Affected table definition
CREATE TABLE [dbo].[Translation]
(
[Locale] NVARCHAR(10) NOT NULL PRIMARY KEY
, [Definition] JSON NOT NULL
)
Corrupt data are in the [Definition] column.
Service: Azure SQL Database
I can provide more examples upon request.
Thank you for fixing this, Microsoft.
Kind regards,
Jozef Sčensný
SWE | DBA | Data Engineer
- Environment (local platform and source/target platforms):
**Steps to Reproduce:**
1.
2.
**Did this occur in prior versions? If not - which version(s) did it work in?**
(DacFx/SqlPackage/SSMS/Azure Data Studio)
Contributor guide
Research direction
Start with the Azure SQL Database Overview blade's Import database action and reproduce the report using the supplied dbo.Translation table and JSON Definition column. Compare the source and restored values for German characters, recording the SqlPackage or DacFx and platform versions; done means the imported JSON preserves the original text.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, sql
- Domain
- cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100