drivendataorg / drivendataorg/cloudpathlib

Methods that move/remove files on the cloud don't remove the cached version

Open
#388 3 comments 0 reactions 0 assignees View on GitHub
bug caching good first issue
Dominant language
Python
Stars
628
Forks
88
Avg merge
17h 28m
Merged PRs (30d)
2

Description

The following code
```python
import cloudpathlib
s3_path = cloudpathlib.S3Path(f"s3://my-bucket/foo.txt")
s3_path.open("x").close() # Create empty file
s3_path.unlink()
s3_path.open("x").close() # Create empty file
```

does not work as expected. I would expect it to create a file, remove it, and then create it again. Instead, I get

```
Traceback (most recent call last):
File "/workspaces/scoutingtool/backend/../cpltest.py", line 9, in
s3_path.open("x").close() # Create empty file
^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/cloudpathlib/cloudpath.py", line 503, in open
buffer = self._local.open(
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/pathlib.py", line 1044, in open
return io.open(self, mode, buffering, encoding, errors, newline)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/tmp/tmpg5cu85sn/my-bucket/foo.txt'
```

This seems to happen because the file is kept in cache, even when it was removed from S3. A simple solution would probably to use the "w" mode, even when "x" was supplied.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.