pingcap / pingcap/tidb-binlog

Parse longtext data type value getting wrong when using reparo to file

Open
#1,019 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
291
Forks
131
Avg merge
5m
Merged PRs (30d)
2

Description

Bug Report

Please answer these questions before submitting your issue. Thanks!

  1. What did you do?
    when column in tidb defined as longtext type,reparo cmd would parse wrong value.

  2. What did you expect to see?
    In mysql client,the real value is 'Oct 23 14:12:06 SVRXXXXHP360 salt-minion: [WARNING ] /usr/lib/python2.7/site-packages/salt/modules/config.py:258: DeprecationWarning: pillar_opts will default to False in the Lithium releasexxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

  3. What did you see instead?
    In reparo cmd using '/data/tidb-binlog-cluster-latest-linux-amd64/bin/reparo',the value changed to blow:
    [79 99 116 32 50 51 32 49 52 58 49 50 58 48 54 32 83 86 82 50 49 54 56 49 72 80 51 54 48 32 115 97 108 116 45 109 105 110 105 111 110 58 32 91 87 65 82 78 73 78 71 32 93 32 47 117 115 114 47 108 105 98 47 112 121 116 104 111 110 50 46 55 47 115 105 116 101 45 112 97 99 107 97 103 101 115 47 115 97 108 116 47 109 111 100 117 108 101 115 47 99 111 110 102 105 103 46 112 121 58 50 53 56 58 32 68 101 112 114 101 99 97 116 105 111 110 87 97 114 110 105 110 103 58 32 112 105 108 108 97 114 95 111 112 116 115 32 119 105 108 108 32 100 101 102 97 117 108 116 32 116 111 32 70 97 108 115 101 32 105 110 32 116 104 101 32 76 105 116 104 105 117 109 32 114 101 108 101 97 115 101 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120 120]

  4. Please provide the relate downstream type and version of drainer.
    (run drainer -V in terminal to get drainer's version)
    Release Version: v4.0.0-beta.1-2-g20397869
    Git Commit Hash: 20397869d35a5bc8a130f615ceb669322fa8600e
    Build TS: 2020-03-02 09:50:48
    Go Version: go1.13
    Go OS/Arch: linux/amd64

5.How to fix it?
Add mysql.TypeLongBlob in reparo/syncer/util.go line 26

-       case mysql.TypeDate, mysql.TypeDatetime, mysql.TypeNewDate, mysql.TypeTimestamp, mysql.TypeDuration, mysql.TypeDecimal, mysql.TypeNewDecimal, mysql.TypeVarchar, mysql.TypeString, mysql.TypeJSON:
+      case mysql.TypeDate, mysql.TypeDatetime, mysql.TypeNewDate, mysql.TypeTimestamp, mysql.TypeDuration, mysql.TypeDecimal, mysql.TypeNewDecimal, mysql.TypeVarchar, mysql.TypeString, mysql.TypeJSON, mysql.TypeLongBlob:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in reparo/syncer/util.go at line 26 and inspect how MySQL column types are handled when reparo writes data to a file. Verify the longtext value is preserved as text rather than converted to a byte-value list, using the reported reparo command and longtext example as the completion check.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.