Making pixels with different colors transparent when comparing 2 images

Hi, I got 2 images with are quite similar, but have a different background color. I want to use these different colors to make all those pixels transparent.

So when I have these 2 images:

the result would be this:

I got this working with Imagick, but don't even know where to start with Sharp/libvips:

convert test-black.png test-white.png -alpha off \
          \( -clone 0,1 -compose difference -composite -negate \) \
          \( -clone 0,2 +swap -compose divide -composite \) \
          -delete 0,1 +swap -compose Copy_Opacity -composite \
          test-transparent.png

Anybody got some ideas how I could do this in Sharp?

Start with Sharp and libvips documentation for comparing two images and deriving transparency, using the provided ImageMagick command as the behavioral reference. Confirm whether the required operations are already exposed; done means a reproducible Sharp approach that turns differing background pixels transparent while preserving the shared image content.