julien-duponchelle / julien-duponchelle/python-mysql-replication
Replaying binlog with dropped columns causes unhandled exception.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 690
- PR merge metrics
- No merged PRs in 30d
Description
Steps to reproduce:
1. create a table with few columns
2. generate some row events
3. drop a column from that table
4. generate some more row events
5. start `pymysqlreplication` replaying the binlog from the position prior to dropping the column
ER:
Working fine.
AR:
Unhandled error:
```
File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/binlogstream.py", line 262, in fetchone
self.__freeze_schema)
File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/packet.py", line 98, in __init__
freeze_schema = freeze_schema)
File "/usr/local/lib/python2.7/dist-packages/pymysqlreplication/row_event.py", line 550, in __init__
column_schema = self.column_schemas[i]
IndexError: list index out of range
```
This is partially solved by 4c48538168f4cd3239563393a29b542cc6ffcf4b, but that doesn't solve the deeper issue at hand which is how the schema for tables is obtained. Schema is always obtained from the current version of `information_schema` no matter how far in the past the RowEvent processed is.
Contributor guide
Research direction
Reproduce the dropped-column scenario, then trace binlogstream.py:262 into packet.py:98 and row_event.py:550, comparing the behavior with commit 4c48538168f4cd3239563393a29b542cc6ffcf4b. Investigate how the replay obtains table schema from information_schema for older RowEvents. Done means replaying events across the column drop without an unhandled IndexError and using schema appropriate to the processed history.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100