Clarification regarding feature depth updates

Hello,
I'd like to use ICE-BA with a PNP-style monocular frontend, in which I have metric camera poses, and landmarks with known depth.

1. I feed the 1st keyframe to ICE-BA with the corresponding IMU measurements, 3D camera pose prior, 2D features and their 3D priors.The LBA optimized camera pose roughly remains the same (as expected), while the 3D landmarks try to minimize the re-projection error in the given frame, hence their 3D position change significantly. This is clear. I verified that the old re-projection errors are in the order of 10^-2 (on the frontend), while the ICE-BA new re-projection errors drop below 10^-5 (only for the frame inserted).
2. Then I go ahead, and add a 2nd frame (or more), let it be either a keyframe or a regular frame, having overlapping feature measurements with the 1st frame. What I notice is that the camera pose again roughly remains the same as the input (as expected), but the updated landmarks' 3D position result in large re-projection error.

How do you explain this latter behavior?

No source file or test entry point is named. Start by tracing ICE-BA's landmark/depth update and reprojection-error handling, then reproduce the one-frame and overlapping two-frame cases described here. Done means determining whether the reported behavior is expected or isolating a reproducible defect.

Written by the indexing model from the issue text.