[Bug] [imap] engine_runningJobMetrics size too long, imap monitoring data is too large
- Dominant language
- Java
- Stars
- 9.7k
- Forks
- 2.4k
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 210
Description
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
In split mode, after MySQL CDC runs for a period of time, the imap is stored on the local disk and takes up too much space, a full 200GB. The parameters configured as follows do not clear the data. If this continues, the task will definitely not be able to run.
seatunnel:
engine:
history-job-expire-minutes: 1440
telemetry:
logs:
scheduled-deletion-enable: true
### SeaTunnel Version
2.3.9
### SeaTunnel Config
```conf
MySQL的cdc
```
### Running Command
```shell
MySQL的cdc
```
### Error Exception
```log
MySQL的cdc
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots

### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by investigating the engine telemetry logs and the history-job-expire-minutes and scheduled-deletion-enable settings described in the issue, then reproduce the MySQL CDC split-mode workload. Done means expired monitoring data is deleted as configured and the local disk no longer grows to excessive size.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql
- Domain
- backend, databases, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100