debezium / debezium/dbz

Custom Converter interface for MySQL DATETIME/TIMESTAMP can not represent full data type range [DBZ-6010]

Open
#744 0 comments 0 reactions 0 assignees View on GitHub
component/mysql-connector migrated-from-jira type/bug
Dominant language
HTML
Stars
6
Forks
8
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from [DBZ-6010](https://issues.redhat.com/browse/DBZ-6010)

In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
h1. Bug report

For bug reports, provide this information, please:
h2. What Debezium connector do you use and what version?

MySQL connector 1.9.7
h2. What behaviour do you expect?

Custom Converter should take an input that differentiates NULL from 0000-00-00 and 0001-01-01 from 0000-00-00.
h2. What behaviour do you see?

NULL and 0000-00-00 (any zero datetime) are all encoded as a zero timestamp.
h2. Do you see the same behaviour using the latest relesead Debezium version?

Yes

 

Here is my custom converter.  Note it can't different between NULL and 0000-00-00 properly.  NULL will be returned as 0000-00-00 by the converter because a zero timestamp is used to represent both values.

 

```

import io.debezium.spi.converter.CustomConverter;
import io.debezium.spi.converter.RelationalColumn;
import org.apache.kafka.connect.data.SchemaBuilder;
import org.apache.kafka.connect.data.Struct;
import java.util.Properties;

public class DatetimeValueConverter implements CustomConverter {

    private SchemaBuilder datetimeSchema;

    @Override
    public void configure(Properties props) {
        datetimeSchema = SchemaBuilder.string().name(props.getProperty("schema.name"));
    }

    @Override
    public void converterFor(RelationalColumn column,
            ConverterRegistration registration) {
        if ("DATETIME".equals(column.typeName()) || "TIMESTAMP".equals(column.typeName()) || "DATE".equals(column.typeName())) {
            registration.register(datetimeSchema, x -> { 
              if(x == null) \{ return null; }
              else if(x.toString().equals("0") || x.toString().equals("1970-01-01T00:00:00Z") || x.toString().equals("1970-01-01T00:00Z")) {
                 //NULL and 0000-00-00 and 0001-00-00 will all hit here
                return("0000-00-00");

              } else return x.toString(); 
            });
        }
    }
}```

Contributor guide

Open the contributing guide

Research direction

The issue names the CustomConverter interface and the MySQL connector but no repository files or tests. Start by tracing how DATETIME, TIMESTAMP, and DATE values reach CustomConverter, then verify that the converter input can distinguish NULL, zero dates, and early valid dates without collapsing them to one value.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka, mysql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.