drivendataorg / drivendataorg/cloudpathlib
Methods that move/remove files on the cloud don't remove the cached version
- Dominant language
- Python
- Stars
- 628
- Forks
- 88
- Avg merge
- 17h 28m
- Merged PRs (30d)
- 2
Description
The following code
```python
import cloudpathlib
s3_path = cloudpathlib.S3Path(f"s3://my-bucket/foo.txt")
s3_path.open("x").close() # Create empty file
s3_path.unlink()
s3_path.open("x").close() # Create empty file
```
does not work as expected. I would expect it to create a file, remove it, and then create it again. Instead, I get
```
Traceback (most recent call last):
File "/workspaces/scoutingtool/backend/../cpltest.py", line 9, in
s3_path.open("x").close() # Create empty file
^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/cloudpathlib/cloudpath.py", line 503, in open
buffer = self._local.open(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/pathlib.py", line 1044, in open
return io.open(self, mode, buffering, encoding, errors, newline)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/tmp/tmpg5cu85sn/my-bucket/foo.txt'
```
This seems to happen because the file is kept in cache, even when it was removed from S3. A simple solution would probably to use the "w" mode, even when "x" was supplied.
Contributor guide
Assessment
This issue has not been assessed yet.