Scrolling away and back causes swiped item to switch direction

Hello, wonderful component.

I am having an issue with the RecyclerView implementation. I have a left and right swipe working but my problem is when I scroll away and come back, the item usually switches to the right drag edge layout even when it was originally the left drag edge layout. If I keep scrolling back and forth, it sometimes becomes the original state until I leave again.

I think I am missing an option or maybe the component likes to default to the right edge swipe, which is what is usually enabled.
Let me know which pieces of code you want to see, cannot post the whole thing.

Edit: Forgot to mention that I am using Attribute.Mode.Multiple and the functionality is for one card to be left edge swiped, then a different one to be right edge swiped. Maintaining both card's swiped state as I scroll around.

Thanks