[BUG]: Extension method JsonSerDe SortProperties removes arrays with strings from metadata during serialization
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 332
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 9
Description
**Describe the bug**
If you include an array of values in a StructField metadata instance they will not serialize when .Json is called on the root StructType. They will however show on the fields JsonValue.
**To Reproduce**
Steps to reproduce the behavior:
```
var values = new[] { "Test1", "Test2" };
var metadata = new JObject(new JProperty("allowedValues", values));
var field = new StructField("TestField", new StringType(), true, metadata);
var fields = new StructType(new[] { field });
var json = fields.Json;
```
json will contain `{"fields":[{"metadata":{"allowedValues":[]},"name":"TestField","nullable":true,"type":"string"}],"type":"struct"}`
**Expected behavior**
I would expect allowedValues in the example above to contain Test1 and Test2.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Microsoft.Spark 2.1.1
**Additional context**
The code that sorts the array in the serialization when you call DataType.Json does not take into account if the item is a simple string and thus returns an empty array:
https://github.com/dotnet/spark/blob/45d0058e315825c905996e87dcaf8bc0feeb64f0/src/csharp/Microsoft.Spark/Interop/Ipc/JsonSerDe.cs#L55-L65
Contributor guide
Assessment
This issue has not been assessed yet.