Controlnet does nothing...

Hey guys!
So I'm having a lot of fun with the plugin but so far I've been totally unable to make controlnet work.
It works fine on its own in the Automatic1111 UI, but simply refuses to do anything at all in photoshop, no matter what I do.
I'm not getting any error in the photoshop UI nor in the command window. The controlnet tab is properly configured with my target image and even shows me the found poses, zdepth and so on. When I start generating an image it shows it is loading controlnet, shows it going up a percent or two and then just... skips it? No matter the settings, the generated images doesn't follow controlnet at all.
I'm at a loss here...
Any idea what I'm doing wrong or how to fix it?

Best
Manny