read() returns incomplete file? (not clear on documentation)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
Hello,
This is my first time using the library, so please apologize me if I'm missing something obvious.
I'm trying to read a .csv.gz file (quite small, less than 100kb) following the documentation (https://gcsfs.readthedocs.io/en/latest/#examples), by doing:
fs = gcsfs.GCSFileSystem(project=project_id, token=service_key_path)
uri = bucket_name + '/' + filename
with fs.open(uri, 'rb') as f:
content = f.read()
Here, I expect content to have the entire file, but it has only 168 rows (out of 946 in total). Reading the same file with google-cloud-storage works fine.
storage_client = storage.Client(credentials=credentials, project=project_id)
bucket = storage_client.bucket(bucket_name)
blob = bucket.blob(filename)
content = blob.download_as_string()
What am I missing?
Thanks
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the documented gcsfs example and the fs.open(..., 'rb').read() entry point described in the report, then compare its result with google-cloud-storage's download_as_string(). Reproduce the .csv.gz case and determine whether the incomplete content is a library bug or a documentation issue; done means the cause and expected behavior are established and covered by a clear fix or documentation update.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 42/100