fsspec / fsspec/filesystem_spec

`SFTPFileSystem.ls()` in root directory hides names

Open
#759 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.4k
Forks
490
Avg merge
2d 3h
Merged PRs (30d)
38

Description

Consider the following methods

@staticmethod
def _decode_stat(stat, parent_path=None):
    if S_ISDIR(stat.st_mode):
        t = "directory"
    elif S_ISLNK(stat.st_mode):
        t = "link"
    else:
        t = "file"
    out = {
        "name": "",
        "size": stat.st_size,
        "type": t,
        "uid": stat.st_uid,
        "gid": stat.st_gid,
        "time": datetime.datetime.utcfromtimestamp(stat.st_atime),
        "mtime": datetime.datetime.utcfromtimestamp(stat.st_mtime),
    }
    if parent_path:
        out["name"] = "/".join([parent_path.rstrip("/"), stat.filename])
    return out

def ls(self, path, detail=False):
    logger.debug("Listing folder %s" % path)
    stats = [self._decode_stat(stat, path) for stat in self.ftp.listdir_iter(path)]
    if detail:
        return stats
    else:
        paths = [stat["name"] for stat in stats]
        return sorted(paths)

And now suppose we want to run fs.ls(""), where "" is the root marker for SFTPFileSystem. The crucial part is here, in _decode_stat. Note that we are calling self._decode_stat(state, parent_path="")

out = {
    "name": "",
    "size": stat.st_size,
    "type": t,
    "uid": stat.st_uid,
    "gid": stat.st_gid,
    "time": datetime.datetime.utcfromtimestamp(stat.st_atime),
    "mtime": datetime.datetime.utcfromtimestamp(stat.st_mtime),
}
if parent_path:  # <-- This evaluates to false when parent_path=""
    out["name"] = "/".join([parent_path.rstrip("/"), stat.filename])

The result is that all the items in the root directory come back as ["", "", "". ...]. The condition should likely be if parent_path is not None:

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at SFTPFileSystem._decode_stat and ls, using the issue's reproduction fs.ls("") to inspect the root-directory listing. Done means the returned names contain each root item rather than repeated empty strings, while preserving the existing detailed and sorted listing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.