How to Crop .PCD Points using X,Y Points

I want to crop the .pcd or .ply points using X,Y points, i can view all the pcd point using numpy array, like:
`from open3d import *
pts = np.asarray(pcd.points)`

after that I have **Boundary** points in another numpy array like this:

> [[448 214] [447 215] [445 215] [444 216] [443 216] [442 217] [441 217] [439 219] [438 219] [438 220] [436 222] [436 223]]

I got these points using contours.
Now i want to crop the object which is in .pcd file X,Y plane is same but the Z values are unknown.

Like the image is showing the pcd file and i want to crop only the Ball, as i have Boundary X,Y values using contour but how can i crop in 3D plane