Orientation issue with equirectangular images

I'm trying to run the reconstruction on a set of sequential equirectangular images recorded with a GoPro MAX on top of a car. No matter what I do, halfway the segment the orientation seems to go upside down during a turn. The images have been pre-filtered so that each is about 1 meter apart.

I thought it was just the frontal direction of the camera being changed as is apparent in this plot:

Note: red dots are the camera origins, green lines/dots the bearing for the middle of the image

Where the orientation of the camera relative to the driving suddenly swaps to the other side, even though it is fixed in reality.

I've also plotted the Euler angles calculated from the rotation matrix and it seems the camera orientation gradually flips upside down during the turn:

This is the original gps trace:

I've tried setting the orientation prior to horizontal, no_roll or naive, but same issue. With or without GPS data, 4k or 16k features, increasing feature image size, increased matches, local_bundle_radius,... none of it seems to have any effect.

Any idea to why this is happening? or where I can look to debug the issue?

This is the full config that I use:

{
    'processes': 32, 
    'matching_time_neighbors': 2,
    'matching_gps_neighbors': 0,
    'matching_gps_distance': 0,
    
    'local_bundle_radius': 1,
    'use_altitude_tag': 'no',
    'align_orientation_prior': 'horizontal',
}

In the exif_overrides I've atm only configured the capture_time and the orientation (equal to 1).

No source files or tests are named. Reproduce the reconstruction with the supplied GoPro MAX equirectangular images and configuration, then trace the orientation handling while comparing the rotation matrix and Euler-angle plots through the turn. Done means identifying why the orientation flips and documenting or correcting the behavior.