Uninstalling seems to remove all of site-packages directory

**Describe the bug**
When uninstalling, instead of just removing its own directory, the whole site-packages directory seems to be removed

**To Reproduce**
Steps to reproduce the behavior:
```
docker run -it 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-training:1.10.0-gpu-py38-cu113-ubuntu20.04-sagemaker bash
root@7568dd42cb42:/# pip uninstall sagemaker-experiments
Found existing installation: sagemaker-experiments 0.1.35
Uninstalling sagemaker-experiments-0.1.35:
Would remove:
/opt/conda/lib/python3.8/site-packages/*
Would not remove (might be manually added):

```
**Expected behavior**
Only remove /opt/conda/lib/python3.8/site-packages/sagemaker_experiments*

**Screenshots**
Log shared above

**Environment:**
Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans):1.10
Framework Version: PT
Python Version: 3.8
CPU or GPU: NA
Python SDK Version: 2.69
Are you using a custom image:No

**Additional context**
Add any other context about the problem here.