confluentinc / confluentinc/libserdes

Double free or corruption C++ Avro

Open
#53 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12
Forks
62
PR merge metrics
No merged PRs in 30d

Description

I need to serialize a JSON string. I used the example code

> ./examples/kafka-serdes-avro-console-producer.cpp

to write a main, which serializes a JSON string as follows.

`{"s":"BTCUSDT","c":"spread","u":123456789,"a":20000.4,"b":21000.5,"A":1.25,"B":0.58,"p":-1,"P":-1,"st":-1,"rt":1675000000123000}`

Here is the code
```

#include "fh_price_t.h"
#include "avro/Encoder.hh"
#include "avro/Decoder.hh"
#include
#include
#include

#include "libserdes/serdescpp-avro.h"

#include
#include
#include
#include

#define FATAL(reason...) do { \
std::cerr << "% FATAL: " << reason << std::endl; \
exit(1); \
} while (0)

template std::string to_string_a (const T& x) {
std::ostringstream oss;
oss << x;
return oss.str();
}

/**
* Convert JSON to Avro Datum.
*
* Returns 0 on success or -1 on failure.
*/
int json2avro (Serdes::Schema *schema, const std::string &json,
avro::GenericDatum **datump) {

avro::ValidSchema *avro_schema = schema->object();

/* Input stream from json string */
std::istringstream iss(json);
auto json_is = avro::istreamInputStream(iss);

/* JSON decoder */
avro::DecoderPtr json_decoder = avro::jsonDecoder(*avro_schema);
avro::GenericDatum *datum = new avro::GenericDatum(*avro_schema);

try {
/* Decode JSON to Avro datum */
json_decoder->init(*json_is);
avro::decode(*json_decoder, *datum);

} catch (const avro::Exception &e) {
std::cerr << "% JSON to Avro transformation failed: "
<< e.what() << std::endl;
return -1;
}

*datump = datum;

}

int main (int argc, char* argv) {

izycoinstestavro::fh_price_t price{};
price.s = "BTCUSDT";
price.c = "spread";
price.u = 123456789;
price.a = 20000.4;
price.A = 1.25;
price.b = 21000.52;
price.B = 0.58;
price.p = -1;
price.P = -1;
price.st = -1;
price.rt = 1675000000123000;

std::ifstream file_ifstream("../src/avro/fh_price_t.json");
std::stringstream buffer;
buffer << file_ifstream.rdbuf();
std::string schema_json = buffer.str();

Serdes::Conf *sconf = Serdes::Conf::create();
Serdes::Schema *schema;
std::string errstr;

if (sconf->set("schema.registry.url", "http://localhost:8081", errstr))
FATAL("Conf failed: " << errstr);

if (sconf->set("serializer.framing", "cp1", errstr))
FATAL("Conf failed: " << errstr);

Serdes::Avro *serdes = Serdes::Avro::create(sconf, errstr);
if (!serdes)
FATAL("Failed to create Serdes handle: " << errstr);

schema = Serdes::Schema::add(serdes, "fh_price_t", schema_json, errstr);
if (!schema)
FATAL("Failed to register schema " << "fh_price_t" << ": " << errstr);

std::cout << "% Registered schema " << schema->name() << " with id " << schema->id() << std::endl;

avro::GenericDatum *datum = NULL;
std::vector out;

/* Convert JSON to Avro object */
std::string line = to_string_a(price);
json2avro(schema, line, &datum);

//std::cout << to_string_a(price) << std::endl;
/*if (rr == -1) {
FATAL("Failed to convert JSON to Avro " << to_string_a(price) << ": " << errstr);
}

// Serialize Avro
if (serdes->serialize(schema, datum, out, errstr) == -1) {
std::cerr << "% Avro serialization failed: " << errstr << std::endl;
delete datum;
exit(1);
}
delete datum;

std::cout << "Data Size : " << out.size() << std::endl;*/

return 0;
}
```

When I run it, I get a double free or corruption (out) error. The error occurs at the output of the function. I have located the line that causes the error (because when I remove it, the error disappears)
`
avro::GenericDatum *datum = new avro::GenericDatum(*avro_schema);`

I would like to know why and how to solve it.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.