debezium / debezium/dbz

Adding element to array causes exception with MongoDB Unwrap SMT [DBZ-710]

Open
#141 0 comments 0 reactions 0 assignees View on GitHub
component/mongodb-connector migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
9
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-710](https://issues.redhat.com/browse/DBZ-710)

I tried to push another element into a document with an array, while using the unwrap SMT and Avro serialization:

{code}
# Fine
db.customers.insert([
{ _id : 1005, first_name : 'Bob', last_name : 'Hopper', email : 'thebob@example.com', address : { street : 'main street', zip : 12345 },
grades : [ 123, 456, 789]
}
]);

{code}

{code}
# Crashes
db.customers.update(
{ _id: 1005 },
{ $push: { grades: 42 } }
)
{code}

{code}
org.apache.avro.SchemaParseException: Illegal character in: grades.3
connect_1 | at org.apache.avro.Schema.validateName(Schema.java:1151)
connect_1 | at org.apache.avro.Schema.access$200(Schema.java:81)
connect_1 | at org.apache.avro.Schema$Field.(Schema.java:403)
connect_1 | at org.apache.avro.SchemaBuilder$FieldBuilder.completeField(SchemaBuilder.java:2124)
connect_1 | at org.apache.avro.SchemaBuilder$FieldBuilder.completeField(SchemaBuilder.java:2116)
connect_1 | at org.apache.avro.SchemaBuilder$FieldBuilder.access$5300(SchemaBuilder.java:2034)
connect_1 | at org.apache.avro.SchemaBuilder$GenericDefault.withDefault(SchemaBuilder.java:2423)
connect_1 | at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:898)
connect_1 | at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:799)
connect_1 | at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:652)
connect_1 | at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:647)
connect_1 | at io.confluent.connect.avro.AvroData.fromConnectData(AvroData.java:324)
connect_1 | at io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:75)
connect_1 | at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:220)
connect_1 | at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:187)
connect_1 | at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:170)
connect_1 | at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:214)
connect_1 | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
connect_1 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
connect_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
connect_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
connect_1 | at java.lang.Thread.run(Thread.java:748)
{code}

Contributor guide

Open the contributing guide

Research direction

The report names no repository file or test path. Start with the MongoDB $push reproduction and the AvroData.fromConnectData/SchemaBuilder stack trace, then trace how the unwrapped array update is converted before Avro serialization. Done means appending an array element no longer throws SchemaParseException in the stated setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, mongodb
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.