Mysql Connector with ProxySQL [DBZ-2667]
- Dominant language
- HTML
- Stars
- 6
- Forks
- 8
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 1
Description
Migrated from [DBZ-2667](https://issues.redhat.com/browse/DBZ-2667)
Hello team,
I am trying to setup a Mysql Connector that connects with Mysql cluster with 2 replica nodes (I want the connector to read data from replicas only) though ProxySQL to achieve high availability -> but it did not work.
I also tried with HAProxy and it worked.
Is that possible to support the connector to connect to ProxySQL (as it's quite popular proxy for Mysql) or we have to go with HAProxy?
Here is the logs when the connector connects though ProxySQL
{code:java}
2020-10-16 07:55:57,073 INFO MySQL|my_server_test|snapshot 'performance_schema.variables_info' is filtered out of capturing [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:57,073 INFO MySQL|my_server_test|snapshot 'performance_schema.variables_info' is filtered out of capturing [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:57,077 INFO MySQL|my_server_test|snapshot including 'sys.sys_config' among known tables [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:57,078 INFO MySQL|my_server_test|snapshot 'sys.sys_config' is filtered out of capturing [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:57,081 INFO MySQL|my_server_test|snapshot snapshot continuing with database(s): [my_db_test] [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:57,081 INFO MySQL|my_server_test|snapshot Step 6: generating DROP and CREATE statements to reflect current database schemas: [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,754 INFO MySQL|my_server_test|snapshot Step 7: releasing global read lock to enable MySQL writes [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,756 INFO MySQL|my_server_test|snapshot Step 7: blocked writes to MySQL for a total of 00:00:01.736 [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,758 INFO MySQL|my_server_test|snapshot Step 8: scanning contents of 0 tables while still in transaction [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,759 INFO MySQL|my_server_test|snapshot Step 8: scanned 0 rows in 0 tables in 00:00:00.0 [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,760 INFO MySQL|my_server_test|snapshot Step 9: committing transaction [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,762 INFO MySQL|my_server_test|snapshot Completed snapshot in 00:00:01.816 [io.debezium.connector.mysql.SnapshotReader]2020-10-16 07:55:58,925 INFO MySQL|my_server_test|task Transitioning from the snapshot reader to the binlog reader [io.debezium.connector.mysql.ChainedReader]2020-10-16 07:55:58,926 WARN MySQL|my_server_test|task After applying blacklist/whitelist filters there are no tables to monitor, please check your configuration [io.debezium.relational.RelationalDatabaseSchema]2020-10-16 07:55:58,943 INFO MySQL|my_server_test|task Creating thread debezium-mysqlconnector-my_server_test-binlog-client [io.debezium.util.Threads]2020-10-16 07:55:58,946 INFO MySQL|my_server_test|task Creating thread debezium-mysqlconnector-my_server_test-binlog-client [io.debezium.util.Threads]Oct 16, 2020 7:55:59 AM com.github.shyiko.mysql.binlog.BinaryLogClient connectINFO: Connected to 172.17.0.11:6033 at mysql-bin.000003/3815 (sid:2, cid:207)2020-10-16 07:55:59,089 INFO MySQL|my_server_test|binlog Connected to MySQL binlog at 172.17.0.11:6033, starting at binlog file 'mysql-bin.000003', pos=3815, skipping 0 events plus 0 rows [io.debezium.connector.mysql.BinlogReader]2020-10-16 07:55:59,089 INFO MySQL|my_server_test|task Waiting for keepalive thread to start [io.debezium.connector.mysql.BinlogReader]2020-10-16 07:55:59,091 INFO MySQL|my_server_test|binlog Creating thread debezium-mysqlconnector-my_server_test-binlog-client [io.debezium.util.Threads]2020-10-16 07:55:59,129 INFO MySQL|my_server_test|binlog Stopped reading binlog after 0 events, no new offset was recorded [io.debezium.connector.mysql.BinlogReader]2020-10-16 07:55:59,191 INFO MySQL|my_server_test|task Keepalive thread is running [io.debezium.connector.mysql.BinlogReader]2020-10-16 07:56:56,381 INFO || WorkerSourceTask{id=my-connector-mysql-test-ncic-0} Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask]2020-10-16 07:56:56,381 INFO || WorkerSourceTask{id=my-connector-mysql-test-ncic-0} flushing 0 outstanding messages for offset commit [org.apache.kafka.connect.runtime.WorkerSourceTask]Oct 16, 2020 7:56:59 AM com.github.shyiko.mysql.binlog.BinaryLogClient$5 runINFO: Trying to restore lost connection to 172.17.0.11:60332020-10-16 07:56:59,093 INFO MySQL|my_server_test|binlog Creating thread debezium-mysqlconnector-my_server_test-binlog-client [io.debezium.util.Threads]2020-10-16 07:56:59,229 INFO MySQL|my_server_test|binlog Connected to MySQL binlog at 172.17.0.11:6033, starting at binlog file 'mysql-bin.000003', pos=3815, skipping 0 events plus 0 rows [io.debezium.connector.mysql.BinlogReader]Oct 16, 2020 7:56:59 AM com.github.shyiko.mysql.binlog.BinaryLogClient connectINFO: Connected to 172.17.0.11:6033 at mysql-bin.000003/3815 (sid:2, cid:208)2020-10-16 07:56:59,269 INFO MySQL|my_server_test|binlog Stopped reading binlog after 0 events, no new offset was recorded [io.debezium.connector.mysql.BinlogReader]2020-10-16 07:57:56,382 INFO || WorkerSourceTask{id=my-connector-mysql-test-ncic-0} Committing offsets [org.apache.kafka.connect.runtime.WorkerSourceTask]2020-10-16 07:57:56,382 INFO || WorkerSourceTask{id=my-connector-mysql-test-ncic-0} flushing 0 outstanding messages for offset commit [org.apache.kafka.connect.runtime.WorkerSourceTask]Oct 16, 2020 7:57:59 AM com.github.shyiko.mysql.binlog.BinaryLogClient$5 runINFO: Trying to restore lost connection to 172.17.0.11:60332020-10-16 07:57:59,231 INFO MySQL|my_server_test|binlog Creating thread debezium-mysqlconnector-my_server_test-binlog-client [io.debezium.util.Threads]Oct 16, 2020 7:57:59 AM com.github.shyiko.mysql.binlog.BinaryLogClient connectINFO: Connected to 172.17.0.11:6033 at mysql-bin.000003/3815 (sid:2, cid:209)2020-10-16 07:57:59,365 INFO MySQL|my_server_test|binlog Connected to MySQL binlog at 172.17.0.11:6033, starting at binlog file 'mysql-bin.000003', pos=3815, skipping 0 events plus 0 rows [io.debezium.connector.mysql.BinlogReader]2020-10-16 07:57:59,405 INFO MySQL|my_server_test|binlog Stopped reading binlog after 0 events, no new offset was recorded [io.debezium.connector.mysql.BinlogReader]
{code}
Contributor guide
Research direction
Reproduce the reported setup with a MySQL cluster, two replica nodes, and ProxySQL, then compare it with the working HAProxy setup. Start with the logged SnapshotReader, BinlogReader, and BinaryLogClient behavior; done means determining whether ProxySQL can be supported and documenting the required behavior or limitations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100