`infer_panorama.py` fails on a specific panorama image

I have encountered that running infer_panorama.py on this panorama image below fails with the following error thrown.

File "/mnt/DataDisk3/satyam/MoGe/scripts/infer_panorama.py", line 273, in main
    output = model.infer(image_tensor, fov_x=fov_x, apply_mask=False)
  File "/home/visualcomputing/anaconda3/envs/MoGe/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/mnt/DataDisk3/satyam/MoGe/moge/model/moge_model.py", line 350, in infer
    _, shift = recover_focal_shift(points, None if mask is None else mask > 0.5, focal=focal)
  File "/mnt/DataDisk3/satyam/MoGe/moge/utils/geometry_torch.py", line 156, in recover_focal_shift
    optim_shift_i = solve_optimal_shift(uv_lr_i_np, points_lr_i_np, focal_np[i])
  File "/mnt/DataDisk3/satyam/MoGe/moge/utils/geometry_numpy.py", line 105, in solve_optimal_shift
    solution = least_squares(partial(fn, uv, xy, z), x0=0, ftol=1e-3, method='lm')
  File "/home/visualcomputing/anaconda3/envs/MoGe/lib/python3.10/site-packages/scipy/optimize/_lsq/least_squares.py", line 851, in least_squares
    raise ValueError("Method 'lm' doesn't work when the number of "
ValueError: Method 'lm' doesn't work when the number of residuals is less than the number of variables.

What could be the reason? I debugged and found out that uv and xyz going into solve_optimal_shift are empty tensors, maybe something to do with the masks output by the head?

Reproduce the failure from scripts/infer_panorama.py:273 and trace the inputs through moge/model/moge_model.py:350 and moge/utils/geometry_torch.py:156 into moge/utils/geometry_numpy.py:105. Inspect why the masks produce empty uv and xyz for the attached panorama; done means the cause is confirmed and the failure has defined handling.