fsspec / fsspec/gcsfs

put() has inconsistent results if you run from Python Console vs in a file.

Open
#517 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
398
Forks
181
Avg merge
2d 10h
Merged PRs (30d)
36

Description

On version gcsfs==2022.11.0

Consider the following code snippet:

import gcsfs
source = 'abc'  # this is a folder - abc/file.txt, abc/subfolder/file.txt
target = 'my_bucket/folder'
fs = gcsfs.GCSFileSystem(consistency="md5")
fs.touch(target + "/.placeholder", truncate=True)  # to create a folder first.
fs.put(source, target, recursive=True)

If I run this in IPython, I end up with my_bucket/folder/file.txt and my_bucket/folder/subfolder/file.txt.
But if I put the exact same code in a python file and run that, I end up with my_bucket/folder/abc.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the example using gcsfs.GCSFileSystem.put with source abc and target my_bucket/folder, first in IPython and then from a Python file. Compare the resulting object paths and trace the differing execution context; done means both forms consistently produce the expected files under my_bucket/folder.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.