PhotoView detect touch outside it's view

I am using photoview inside my dialogfragment, which have relative layout as it's parents and photoview as it's child. But when I touch my finger on photoview and move up ( outside photoview bound ) and move left or right, image also move accordingly. I think that should not be happen.