julien-duponchelle / julien-duponchelle/python-mysql-replication

Invalid json format

Open
#233 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.4k
Forks
690
PR merge metrics
No merged PRs in 30d

Description

I have a table to store events for publishing to kafka. I use binlog to capture new inserted rows. My table has a payload column to store json data. My application has a bug, it inserts invalid format json payload. The library python-mysql-replication crash when decodes this inserted event. Below is the event of binlog:

[binlog_event.txt](https://github.com/noplay/python-mysql-replication/files/1401772/binlog_event.txt)
```
BEGIN
/*!*/;
# at 241344758
#171020 17:31:02 server id 243 end_log_pos 241344859 CRC32 0x01ef0fcc Table_map: `tala_migration`.`event_store` mapped to number 404
# at 241344859
#171020 17:31:02 server id 243 end_log_pos 241347512 CRC32 0x3becbfcf Write_rows: table id 404 flags: STMT_END_F
### INSERT INTO `tala_migration`.`event_store`
### SET
### @1=21879980
### @2='be8f063b-4a0a-4901-a441-f2efc25913e5'
### @3='order.update'
### @4='tala.order_updated'
### @5='11493761'
### @6=NULL
### @7=NULL
### @8='11493761'
### @9='{"id":"8228025","quote_id":"99997930","original_code":null,"relation_code":null,"code":"11493761","barcode":"11493761","state":null,"substate":null,"status":"giao_hang_thanh_cong","substatus":null,"previous_status":null,"payment_status":1,"subtotal":"99000.00","grand_total":"99000.00","relation_child_id":null,"out_stock_date":null,"sub_cancel_reason":null,"cancel_date":null,"order_type":null,"dropship_already":"0","created_at":"2017-10-20 17:31:26","updated_at":"2017-10-20 17:31:30","purchased_at":"2017-10-20 17:31:30","visitor_session_id":"8agavvc6hujt4ok4u45ps20402","items_count":"1","items_qty":"1","checkout_method":"register","customer_id":"501","customer_email":"banmaixanhjack@gmail.com","customer_full_name":"Hu\x5cu1ef3nh Minh Khai","customer_birthdate":null,"customer_note":null,"customer_ip":null,"tax_company_code":null,"tax_company_name":null,"tax_company_address":null,"shipping_plan_id":"3","shipping_amount":"0","shipping_time_min":"0","shipping_time_max":"1","delivery_commitment_time":"2017-10-21 10:00:00","applied_rule_ids":"733198,733196,733171,733195","applied_rule_ids2":null,"coupon_code":null,"affiliate_code":null,"gift_card_code":null,"gift_card_amount":"0.00","discount_amount":"0.00","discount_other":"0.00","discount_tikier":"0.00","discount_tiki_first":"0.00","discount_tikixu":"0.00","discount_coupon":"0.00","discount_visa":"0.00","discount_hsbc":null,"discount_acbvisa":"0.00","is_cover":"0","tikixu_point":"0","tikixu_point_earning":"0","magento_id":null,"is_24h":"0","is_tiki_first":"1","is_rma":"0","backend_id":null,"warehouse_id":"4","warehouse_data":"{\x5c"warehouse_distribute_data\x5c":{\x5c"warehouse_data\x5c":{\x5c"normal\x5c":{\x5c"4\x5c":{\x5c"stock_items\x5c":{\x5c"52359640\x5c":\x5c"1\x5c"}}}},\x5c"outstock_data\x5c":[]},\x5c"stock_data\x5c":{\x5c"52359640\x5c":{\x5c"stock_data\x5c":{\x5c"2\x5c":{\x5c"is_in_stock\x5c":1,\x5c"qty_salable\x5c":9223372036854775807,\x5c"is_valid_supplier_time\x5c":1,\x5c"is_ready_to_process\x5c":0},\x5c"4\x5c":{\x5c"is_in_stock\x5c":1,\x5c"qty_salable\x5c":9223372036854775807,\x5c"is_valid_supplier_time\x5c":1,\x5c"is_ready_to_process\x5c":0}},\x5c"stock_type\x5c":\x5c"1\x5c",\x5c"qty_ordered\x5c":\x5c"1\x5c",\x5c"product_id\x5c":582522,\x5c"not_allow_transport_by_plane\x5c":false,\x5c"is_bulky\x5c":\x5c"0\x5c"}},\x5c"stock_class_data\x5c":\x5c"Tala\x5c\x5c\x5c\x5cDelivery\x5c\x5c\x5c\x5cOrderRouting\x5c\x5c\x5c\x5cInStock\x5c",\x5c"partner_time\x5c":{\x5c"min\x5c":\x5c"2\x5c",\x5c"max\x5c":\x5c"2\x5c"},\x5c"holiday_time\x5c":0,\x5c"is_new_logic\x5c":1}","platform":"frontend-desktop","extra":null,"type":null,"shipping_discount_amount":"29000.00","shipping_discount_tiki_first":"0.00","customer_backend_id":"501","customer_group_id":"12","items_collection":{}}'
### @10=NULL
### @11=NULL
### @12=0
### @13=0
### @14=NULL
### @15='0000-00-00 00:00:00.000'
### @16='0000-00-00 00:00:00.000'
# at 241347512
#171020 17:31:02 server id 243 end_log_pos 241347543 CRC32 0x9ce30b05 Xid = 32706211
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.