fsspec / fsspec/filesystem_spec

fsspec.fuse ready_file does not work

Open
#1,587 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
490
Avg merge
2d 3h
Merged PRs (30d)
38

Description

Create test file:

echo bar > foo
tar -cf foo{.tar,}

Mount:

from fsspec.implementations.tar import TarFileSystem as tafs
fs = tafs("foo.tar")
import fsspec.fuse
fsspec.fuse.run(fs, "./", "mounted", ready_file=True)

Calling stat and cat on mounted/.fuse_ready will result in Input/output error.

The reason for it not working becomes clear fast when looking into the source code. Namely, FUSEr.getattr returns simply {"type": "file", "st_size": 5}, which is not what fusepy expects. This patch fixes the issue for me:

--- /home/user/.local/lib/python3.10/site-packages/fsspec/fuse.py	2024-04-21 18:49:33.897399229 +0200
+++ /home/user/.local/lib/python3.10/site-packages/fsspec/fuse.py	2024-04-21 18:49:11.905377657 +0200
@@ -1,4 +1,5 @@
 import argparse
+import io
 import logging
 import os
 import stat
@@ -26,13 +27,13 @@
     def getattr(self, path, fh=None):
         logger.debug("getattr %s", path)
         if self._ready_file and path in ["/.fuse_ready", ".fuse_ready"]:
-            return {"type": "file", "st_size": 5}
-
-        path = "".join([self.root, path.lstrip("/")]).rstrip("/")
-        try:
-            info = self.fs.info(path)
-        except FileNotFoundError:
-            raise FuseOSError(ENOENT)
+            info = {"type": "file", "size": 5}
+        else:
+            path = "".join([self.root, path.lstrip("/")]).rstrip("/")
+            try:
+                info = self.fs.info(path)
+            except FileNotFoundError:
+                raise FuseOSError(ENOENT)
 
         data = {"st_uid": info.get("uid", 1000), "st_gid": info.get("gid", 1000)}
         perm = info.get("mode", 0o777)
@@ -70,10 +71,6 @@
 
     def read(self, path, size, offset, fh):
         logger.debug("read %s", (path, size, offset))
-        if self._ready_file and path in ["/.fuse_ready", ".fuse_ready"]:
-            # status indicator
-            return b"ready"
-
         f = self.cache[fh]
         f.seek(offset)
         out = f.read(size)
@@ -97,6 +94,12 @@
 
     def open(self, path, flags):
         logger.debug("open %s", (path, flags))
+
+        if self._ready_file and path in ["/.fuse_ready", ".fuse_ready"]:
+            self.cache[self.counter] = io.BytesIO(b"ready")
+            self.counter += 1
+            return self.counter - 1
+
         fn = "".join([self.root, path.lstrip("/")])
         if flags % 2 == 0:
             # read

I would recommend adding unit tests for all fsspec.fuse features.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the fsspec.fuse example in the issue, then inspect fsspec/fuse.py, especially FUSEr.getattr, open, and read. Add focused tests for the ready_file behavior and confirm that stat and cat on mounted/.fuse_ready succeed and return the expected ready content.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
operating-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.