fsspec / fsspec/filesystem_spec
UnsupportedOperation error with ZipFileSystem using append mode
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 490
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 38
Description
fsspec.implementations.zip.ZipFileSystem seems to have an issue with append mode 'a'. I assume, opening in mode 'a' allows for adding new Zip-entries to an existing Zip archive. At least this is what I need. While opening in append mode first seems to work - I can write to the archive - it will fail once the filesystem is closed.
While using mode 'w' works and creates an empty archive like so:
fs = fsspec.filesystem("zip", fo="test-w.zip", mode="w")
# write stuff here or do nothing...
fs = None # <-- force garbage collection of fs object
An error is raised when using mode 'a' once the filesystem is auto-closed by setting the reference to None or by calling fs.close() directly:
fs = fsspec.filesystem("zip", fo="test-a.zip", mode="a")
# write stuff here or do nothing...
fs = None # <-- error, also for fs.close()
Traceback:
Exception ignored in: <function ZipFileSystem.__del__ at 0x7fa07d5e4fe0>
Traceback (most recent call last):
File ".../lib/python3.11/site-packages/fsspec/implementations/zip.py", line 73, in __del__
self.close()
File ".../lib/python3.11/site-packages/fsspec/implementations/zip.py", line 78, in close
self.zip.close()
File ".../lib/python3.11/zipfile.py", line 1892, in close
self._write_end_record()
File ".../lib/python3.11/zipfile.py", line 1989, in _write_end_record
self.fp.truncate()
io.UnsupportedOperation: truncate
Reproducible on Linux and Windows.
Note, Python docs on zipfile.ZipFile state: If mode is 'a' and the file does not exist at all, it is created. If mode is 'r' or 'a', the file should be seekable.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fsspec/implementations/zip.py, especially ZipFileSystem.del and close, and reproduce the provided append-mode example on close. Compare the append and write paths and verify that closing an archive opened with mode 'a' no longer raises io.UnsupportedOperation: truncate while still allowing entries to be added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100