confluentinc / confluentinc/libserdes

avro::decode not able to handle NULL array. Thows "Incorrect token in the stream. Expected: Array start, found Null"

Open
#49 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12
Forks
62
PR merge metrics
No merged PRs in 30d

Description

The serdes CPP code is not able to handle NULL arrays when converting json to avro.

sample excerpt of input json that fails:
,"list1":[{"test1", "test2"}],"list2":null,"listItemCount":0,.........

It bails out at "avro::decode(*json_decoder, *datum);" while handling list2. How can we make decode accept null values which could be valid in some cases. Same question applies to a record that could be null too. Thanks.

static int
Json2Avro(Serdes::Schema *schema, const std::string &json, avro::GenericDatum **datump)
{
avro::ValidSchema *avro_schema = schema->object();

std::istringstream iss(json);
auto json_is = avro::istreamInputStream(iss);
avro::DecoderPtr json_decoder = avro::jsonDecoder(*avro_schema);
avro::GenericDatum *datum = new avro::GenericDatum(*avro_schema);

try {
json_decoder->init(*json_is);
avro::decode(*json_decoder, *datum);
}
catch (const avro::Exception &ex) {
cerr << "JSON to avro transformation failed: " << ex.what() << endl;
return -1;
}

*datump = datum;

return 0;
}

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the Json2Avro entry point and inspect how avro::jsonDecoder and avro::decode process list2 when the JSON value is null. Determine the expected behavior for nullable arrays and records, then verify that valid null values no longer fail while existing non-null decoding still works.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.