RGBD region of interest

A method to crop the RGBD image to a region of interest. Not just min/max depth.

Could be something like a 2D pixel polygon or rectangle plus a maximum depth plane that can be rotated around x and y axis.

I need it to filter away background with depth that is not perpendicular to the sensor viewing direction.
This background 'noise' can of cause be filtered away in the final model, but it is about 90% of the triangles, and it significantly slows down the computation time, and requires a lot of hardware power. The background is moving and is also noise for the registration process. Any suggestions?