AMP4Email content jumping on changing direction in amp-carousel (gmail iOS)

### Description

This problem is relevant in gmail iOS app:
If you change the direction of scrolling slides in a carousel, you will jump to another part of email

Sample: https://www.awesomescreenshot.com/video/15651410?key=e6deb9078b063501b551076f9562e2ee
We clicked 4 times on right button and one times on left button, after that there was a jump to the random part of email.

It was testing on different version gmail app and iOS. Relevant on amp-version 0.1 and 0.2.

Html code of this letter: [code.zip](https://github.com/ampproject/amphtml/files/10988405/code.zip)

### Reproduction Steps

1. Click on the right arrow in carousel few times
2. Click on the left arrow in carousel.
It's actual also another direction:
1. Click on the left arrow in carousel few times
2. Click on the right arrow in carousel

### Relevant Logs

_No response_

### Browser(s) Affected

_No response_

### OS(s) Affected

_No response_

### Device(s) Affected

_No response_

### AMP Version Affected

0.1, 0.2

Start with the attached code.zip reproduction and the listed carousel interaction steps in Gmail on iOS. Reproduce both right-then-left and left-then-right direction changes, then inspect amp-carousel behavior to identify why the email jumps; done means neither sequence moves to another part of the email.

Written by the indexing model from the issue text.