dotnet / dotnet/spark

[BUG]: UDF returning json as string changing the decimal format inside json

Open
#984 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

**Describe the bug**
UDF returning json as string, but when reading it in main function, decimal format is getting change

sample json returned by UDF

```
{
"Organization":"Org",
"Employee":
{
"Id": 2345678945.1234512 // this is decimal in json but while reading from UDF, decimal part is not coming correctly
"name" : "somename"
}
}
```

```
//UDF with signature
string myFunc(string)
{
return json.serialize(myObject);
}
```

```
//myObject is complex object and looks like
Class myObject
{
string Organization;
Employee Emp;
}

class Employee
{
decimal Id; //if i change it to string, i get full value
string name;
}
```

In my main function i am calling the UDF as
```
df = df.Select(myFunc(col('col1')).As(jsonResult));

// json schema
schema = {StructField(Organization, StringType),
StructFiled(Employee, StringType)} // reading json as String type

//explode json
df = df.Select(FromJson(Col("jsonResult"), schema).As("Result"))

df.Select("Result.*").

df.Select("Employee") // reading json attribute as string, but decimal is getting truncated.
```

If i change my decimal to string value in json, then i am getting the full decimal value. Which means UDF is returning the correct value, but while reading in main function it is truncating decimal (even though i am reading as string).

Thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.