PermissionError on download with objaverse-xl files

Using the `download_objects()` function results in the error _**PermissionError: [Errno 13] Access is denied**_

After debugging a bit through the objaverse package code, it turned out that the problem is in line 322 of the `github.py` file:
```
shutil.rmtree(os.path.join(target_directory, ".git"))
```

Seems like the `.git` folder is not accessible at this point in time. Not sure if this is a windows-specific or maybe even a os-settings-specific issue is, but it prevents me from using the objaverse package without modifying it, so I wanted to report it here.