Changing zoom mode (drag) resets the zoom level

I am using a chart with this zoom configuration (pan is disabled):
```
zoom: {
zoom: {
mode: "xy",
drag: {
enabled: true,
treshold: 100,
drawTime: "afterDatasetsDraw",
},
}
}
```
I noticed that after a first zoom if I try to change the mode, for example with `chart.config.options.plugins.zoom.zoom.mode = "y"`, the mode is succesfully changed, but the zoom level gets resetted.
I made a [sandbox](https://codesandbox.io/p/sandbox/zoomchartjsbug-5mqxlj?file=%2Fsrc%2Fcomponents%2FScatter.vue%3A165%2C98&layout=%257B%2522sidebarPanel%2522%253A%2522EXPLORER%2522%252C%2522rootPanelGroup%2522%253A%257B%2522direction%2522%253A%2522horizontal%2522%252C%2522contentType%2522%253A%2522UNKNOWN%2522%252C%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522id%2522%253A%2522ROOT_LAYOUT%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522UNKNOWN%2522%252C%2522direction%2522%253A%2522vertical%2522%252C%2522id%2522%253A%2522clul7502y0006356i06mq26h3%2522%252C%2522sizes%2522%253A%255B100%252C0%255D%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522EDITOR%2522%252C%2522direction%2522%253A%2522horizontal%2522%252C%2522id%2522%253A%2522EDITOR%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL%2522%252C%2522contentType%2522%253A%2522EDITOR%2522%252C%2522id%2522%253A%2522clul7502y0002356ivgw35jf0%2522%257D%255D%257D%252C%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522SHELLS%2522%252C%2522direction%2522%253A%2522horizontal%2522%252C%2522id%2522%253A%2522SHELLS%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL%2522%252C%2522contentType%2522%253A%2522SHELLS%2522%252C%2522id%2522%253A%2522clul7502y0003356iubx94386%2522%257D%255D%252C%2522sizes%2522%253A%255B100%255D%257D%255D%257D%252C%257B%2522type%2522%253A%2522PANEL_GROUP%2522%252C%2522contentType%2522%253A%2522DEVTOOLS%2522%252C%2522direction%2522%253A%2522vertical%2522%252C%2522id%2522%253A%2522DEVTOOLS%2522%252C%2522panels%2522%253A%255B%257B%2522type%2522%253A%2522PANEL%2522%252C%2522contentType%2522%253A%2522DEVTOOLS%2522%252C%2522id%2522%253A%2522clul7502y0005356i0rn55u8x%2522%257D%255D%252C%2522sizes%2522%253A%255B100%255D%257D%255D%252C%2522sizes%2522%253A%255B50%252C50%255D%257D%252C%2522tabbedPanels%2522%253A%257B%2522clul7502y0002356ivgw35jf0%2522%253A%257B%2522id%2522%253A%2522clul7502y0002356ivgw35jf0%2522%252C%2522tabs%2522%253A%255B%257B%2522id%2522%253A%2522clv3yuwis002p356iwr89gofm%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522type%2522%253A%2522FILE%2522%252C%2522initialSelections%2522%253A%255B%257B%2522startLineNumber%2522%253A8%252C%2522startColumn%2522%253A2%252C%2522endLineNumber%2522%253A8%252C%2522endColumn%2522%253A2%257D%255D%252C%2522filepath%2522%253A%2522%252Fpackage.json%2522%252C%2522state%2522%253A%2522IDLE%2522%257D%252C%257B%2522id%2522%253A%2522clv4v7eer0002356iuo04ert0%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522type%2522%253A%2522FILE%2522%252C%2522initialSelections%2522%253A%255B%257B%2522startLineNumber%2522%253A165%252C%2522startColumn%2522%253A98%252C%2522endLineNumber%2522%253A165%252C%2522endColumn%2522%253A98%257D%255D%252C%2522filepath%2522%253A%2522%252Fsrc%252Fcomponents%252FScatter.vue%2522%252C%2522state%2522%253A%2522IDLE%2522%257D%255D%252C%2522activeTabId%2522%253A%2522clv4v7eer0002356iuo04ert0%2522%257D%252C%2522clul7502y0005356i0rn55u8x%2522%253A%257B%2522tabs%2522%253A%255B%257B%2522id%2522%253A%2522clul7502y0004356irphgy72h%2522%252C%2522mode%2522%253A%2522permanent%2522%252C%2522type%2522%253A%2522UNASSIGNED_PORT%2522%252C%2522port%2522%253A0%252C%2522path%2522%253A%2522%252F%2522%257D%255D%252C%2522id%2522%253A%2522clul7502y0005356i0rn55u8x%2522%252C%2522activeTabId%2522%253A%2522clul7502y0004356irphgy72h%2522%257D%252C%2522clul7502y0003356iubx94386%2522%253A%257B%2522tabs%2522%253A%255B%255D%252C%2522id%2522%253A%2522clul7502y0003356iubx94386%2522%257D%257D%252C%2522showDevtools%2522%253Atrue%252C%2522showShells%2522%253Afalse%252C%2522showSidebar%2522%253Atrue%252C%2522sidebarPanelSize%2522%253A15%257D) showcasing this issue. In the example, try to change the zoom mode, zoom out, then change the zoom mode again to see the issue.
Is this intended? And if so, is there a workaround for this?
Thanks in advance