shimat / shimat/opencvsharp_blazor_sample

Panorama Stitching: Stitcher.Stitch crashes with cv::gemm assertion (CV_64FC2 type mismatch) after upgrading to 5.0.0.20260712

Open
#12 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
HTML
Stars
12
Forks
2
Avg merge
1d 5h
Merged PRs (30d)
4

Description

Summary

After upgrading to OpenCvSharp5/OpenCvSharp5.runtime.wasm 5.0.0.20260712 (which includes the OpenCL fix from opencvsharp#2041), the /samples/aruco page's OpenCL/UMat crash is confirmed resolved. However, /samples/stitching's Cv2.Stitcher.Stitch now fails with a different crash - no longer the OpenCL probe, but an assertion failure deep in the bundle adjustment step:

Uncaught (in promise) RuntimeError
cv::Exception: OpenCV(5.0.0) /home/runner/work/opencvsharp/opencvsharp/opencv-5.0.0/modules/core/src/matmul.simd.hpp:1053:
error: (-215:Assertion failed) type == CV_64FC2 in function 'gemmImpl'

    at cv::error(cv::Exception const&)
    at cv::cpu_baseline::gemmImpl(cv::Mat, cv::Mat, double, cv::Mat, double, cv::Mat, int)
    at cv::cpu_baseline::callGemmImpl<double>(...)
    at cv::gemm(cv::_InputArray const&, cv::_InputArray const&, double, cv::_InputArray const&, double, cv::_OutputArray const&, int)
    at cv::mulTransposed(cv::_InputArray const&, cv::_OutputArray const&, bool, cv::_InputArray const&, double, int)
    at cv::LevMarqDenseLinearBackend::calcFunc(double&, bool, bool)
    at cv::detail::LevMarqBase::optimize()

followed immediately by a second, unrecoverable trap:

RuntimeError: unreachable
    at __trap
    at abort
    at assert
    at runtimeKeepalivePop

Where this happens

PanoramaStitching.razor's StitchAsync:

using var stitcher = Stitcher.Create(Stitcher.Mode.Panorama);
using var pano = new Mat();
var result = stitcher.Stitch(new[] { leftMat, rightMat }, pano);

leftMat/rightMat are two crops of the same 256x256 Mandrill.bmp (180px and 156px wide respectively) sharing an ~80px overlapping band - a deliberately small, synthetic "two photos of the same scene" pair, standing in for a real photo pair a user hasn't uploaded yet.

Question / ask

I don't yet know whether this is:

  • a genuine bug in OpenCV's LevMarqDenseLinearBackend/bundle-adjustment code (a real type mismatch building the parameter matrix passed into gemm), which would need to be reported to opencv/opencv instead, or
  • a usage issue on my end - e.g. the crop pair being too small / too little true parallax for the default bundle adjuster, or Stitcher.Mode.Panorama/the default bundle adjuster needing different settings for such small inputs

Filing here first since I'm not confident yet it's a core algorithm bug rather than something about how this sample is calling Stitcher. If it turns out to reproduce with a normal-sized, real photo pair (not just this synthetic small-crop case), I'll follow up with a report against opencv/opencv directly. Any pointers on which it is - or whether this reproduces outside wasm at all - would be appreciated.

Environment

  • OpenCvSharp5 / OpenCvSharp5.runtime.wasm 5.0.0.20260712
  • Confirmed the OpenCL/UMat crash (opencvsharp#2037) is fixed as of this version - this is a separate, newly-surfaced issue

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with BlazorApp/Pages/PanoramaStitching.razor and its StitchAsync method, then reproduce /samples/stitching with the two Mandrill.bmp crops described. Compare the wasm behavior with a normal-sized, real photo pair and, if possible, outside wasm. Done means establishing whether this is a synthetic-input or wasm-specific issue, or enough evidence for an upstream OpenCV report.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, wasm
Domain
computer-vision, web-dev
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.