julien-duponchelle / julien-duponchelle/python-mysql-replication
Unable to detect log changes after running for a long time.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 690
- PR merge metrics
- No merged PRs in 30d
Description
### Description
When I run a script that monitors binlog changes for a long time, the stream object may not be able to pull the relevant events until I restart the script.
Here is some of my code:
```python
log_pos, log_file = "mysql-bin.000004", 928361
stream = BinLogStreamReader(
connection_settings=connection_settings,
server_id=server_id,
only_events=[WriteRowsEvent, UpdateRowsEvent, DeleteRowsEvent],
only_schemas=["dbname"],
only_tables=['table1', 'table2'],
blocking=True,
resume_stream=True,
log_pos=log_pos,
log_file=log_file
)
# start to listen
for event in stream:
...
```
When the script first runs, it can listen to all log information.
However, after running for a while, for example, 30 minutes or an hour, this listener will stop working(But it won't report an error). In other words, when the data in the relevant table changes, the for loop will not respond. It only works again after I restart the script.
### Related Information
- Platform: Ubuntu Or Windows 11
- MySQL Version: 5.7
- Python Version: 3.11
- python-mysql-replication Version: 0.46
Contributor guide
Research direction
Start with the supplied BinLogStreamReader configuration and reproduce the silent stop after a long-running session on MySQL 5.7. Inspect the blocking stream behavior around periods without matching events and compare it with the restart case. Done means relevant table changes are delivered continuously without restarting the script.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mysql, python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100