Incorrect formatting of optional values

Nobody has claimed this yet.

My Sample code to convert the JSON String(value) to Avro Binary is as follows:

````
native, _, err := schema.Codec().NativeFromTextual([]byte(value))
if err != nil {
return nil, err
}

valueBytes, err := schema.Codec().BinaryFromNative(nil, native)
if err != nil {
return nil, err
}
````

I have the following JSON Schema:
**Schema-1:**
This schema as per Avro specification, needs a mandatory `username` but optional `colour` & `mass` fields which defaults to `null`.
````
{
"type": "record",
"name": "user",
"fields": [
{
"name": "username",
"type": "string"
},
{
"name": "colour",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "mass",
"type": [
"null",
"int"
],
"default": null
}
]
}
````

My incoming JSON String is:
**Ex-1:**
````
{"username":"temp"}
````
This will be converted to Avro just fine. No issues. And the downstream systems like databases considers mass & colour as `null` for this record which is all good.

**Ex-2:**
````
{"username":"temp","colour":"red","mass":10}
````
This fails with the following error:
````
cannot decode textual record \"user\": cannot decode textual union: expected: '{'; actual: '\"' for key: \"colour\"", "message": "{\"username\":\"temp\",\"colour\":\"red\",\"mass\":10}"
````

I couldn't find a way to get around this. However, reading through multiple issues in this repo and one among them is: https://github.com/linkedin/goavro/issues/114, I made the schema change to be as follows:
**Schema-2:**
This schema specifies that `username` is mandatory. However, `colour` & `mass` are optional fields with default values to be `""`(empty string with size 0) & `0`(integer 0) respectively.
````
{
"type": "record",
"name": "user",
"fields": [
{
"name": "username",
"type": "string"
},
{
"name": "colour",
"type": "string",
"default": "" # A null value here fails `NativeFromTextual` with a `panic: runtime error: invalid memory address or nil pointer dereference` error
},
{
"name": "mass",
"type": "int",
"default": 0 # A null value here fails `NativeFromTextual` with a `panic: runtime error: invalid memory address or nil pointer dereference` error
}
]
}
````
My incoming JSON String is:
**Ex-1:**
````
{"username":"temp"}
````
This will be converted to Avro just fine. However, the meaning of the record has now changed due to the defaults not being accepted as null but some values. When the downstream big data systems consume these messages and perform the aggregates the meanings will changed as the int null is different from int 0 and similar with any other data types.

**Ex-2:**
````
{"username":"temp","colour":"red","mass":10}
````
This works just fine. No issues

What I wanted is:
1. A schema that works for both the below JSON messages with out changing the meaning of the message i.e the schema should accept null values when the incoming messages doesn't have the associated fields defined.
````
{"username":"temp"}
````
````
{"username":"temp","colour":"red","mass":10}
````
Please note that, I do not intend to change the JSON message structure.

Any inputs are greatly appreciated.

Start by reproducing the two JSON inputs with Schema-1 through NativeFromTextual and BinaryFromNative, then compare the behavior with Schema-2. The issue is resolved when optional fields can be omitted or supplied with values without changing their intended null semantics, with the observed errors covered by tests.

Written by the indexing model from the issue text.