[FEATURE] I need MORE CONTROL over outpainting. Can you make a ControlNet level of control for OUTpainting (using an input image)

Hello,
Is it possible to use an input image as a reference for controling how the outpaining image will be?
Let me give an example, let's take this image:

I would like to outpaint the right side but I want to use an image to control what will be outpainted:

So the input image I have to use is this one:

Same thing with this example:

Let's suppose I did not have any idea what is the rest of the image, then we can input an image to control what is the outpaining gonna do:

Outpainted image:

The actual outpaint method you described with the use of the llama thing is random and has not control of what is going to be outpainted except for the prompt and usual parameters.

Can you make something like what I described?
Thanks
@lllyasviel

No files, tests, or entry points are named. Start by locating the existing outpainting implementation and reviewing how its input image, mask, prompt, and control parameters are handled. Done would mean defining and implementing reference-image control for outpainting, with tests or examples showing that the requested region follows the supplied image.