julien-duponchelle / julien-duponchelle/python-mysql-replication

Unable to detect log changes after running for a long time.

Open
#636 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.4k
Forks
690
PR merge metrics
No merged PRs in 30d

Description

### Description
When I run a script that monitors binlog changes for a long time, the stream object may not be able to pull the relevant events until I restart the script.

Here is some of my code:
```python
log_pos, log_file = "mysql-bin.000004", 928361
stream = BinLogStreamReader(
connection_settings=connection_settings,
server_id=server_id,
only_events=[WriteRowsEvent, UpdateRowsEvent, DeleteRowsEvent],
only_schemas=["dbname"],
only_tables=['table1', 'table2'],
blocking=True,
resume_stream=True,
log_pos=log_pos,
log_file=log_file
)

# start to listen
for event in stream:
...
```
When the script first runs, it can listen to all log information.
However, after running for a while, for example, 30 minutes or an hour, this listener will stop working(But it won't report an error). In other words, when the data in the relevant table changes, the for loop will not respond. It only works again after I restart the script.

### Related Information
- Platform: Ubuntu Or Windows 11
- MySQL Version: 5.7
- Python Version: 3.11
- python-mysql-replication Version: 0.46

Contributor guide

Open the contributing guide

Research direction

Start with the supplied BinLogStreamReader configuration and reproduce the silent stop after a long-running session on MySQL 5.7. Inspect the blocking stream behavior around periods without matching events and compare it with the restart case. Done means relevant table changes are delivered continuously without restarting the script.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.