sys:1: RuntimeWarning: coroutine 'GCSFileSystem._info' was never awaited
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 398
- Forks
- 181
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 36
Description
What happened:
Exception:
Traceback (most recent call last):
File "googleload.py", line 12, in <module>
asyncio.get_event_loop().run_until_complete(main())
File "/path/to/python/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "googleload.py", line 9, in main
async with await fs.open("/path/to/file.txt") as fp:
File "/path/to/python/lib/python3.8/site-packages/fsspec/spec.py", line 838, in open
f = self._open(
File "/path/to/python/lib/python3.8/site-packages/gcsfs/core.py", line 1141, in _open
return GCSFile(
File "/path/to/python/lib/python3.8/site-packages/gcsfs/core.py", line 1265, in __init__
super().__init__(
File "/path/to/python/lib/python3.8/site-packages/fsspec/spec.py", line 1135, in __init__
self.size = self.details["size"]
TypeError: 'coroutine' object is not subscriptable
ERROR:asyncio:Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x10a6ad6a0>
sys:1: RuntimeWarning: coroutine 'GCSFileSystem._info' was never awaited
What you expected to happen:
I'm not 100% sure how to use the new async part, as it is quite poorly documented so far.
I don't even know which part of open/read/with needs to be awaited/asynced.
But no matter what I choose there, it fails because the exception already fails in fs.open.
I suspect that self.details is set to the return value of some function, and in the original sync code the return value was just the details.
As you can see in line 1134 of fsspec's specy.py:
self.details = fs.info(path)
I'm guessing that fs.info is a coroutine in the async implementation, therefore assigns an Awaitable to self.details.
Minimal Complete Verifiable Example:
import asyncio
import gcsfs
async def main():
loop = asyncio.get_event_loop()
fs = gcsfs.GCSFileSystem(project="my-project", asynchronous=True, loop=loop)
await fs.set_session()
async with await fs.open("/path/to/file.txt") as fp:
print(await fp.read())
asyncio.get_event_loop().run_until_complete(main())
What I would like to know:
Could you also provide me with the way how the opened file-pointer is supposed to be used afterwards?
Unfortunately, you don't have any documentation beyond that I need to await the session in the beginning.
Do I await on fs.open, or is fs.open itself sync? Do I await fp.read or is everything buffered on open already anyway and the actual read can be sync because it just reads from memory?
Environment:
- gcsfs version: 0.7.0
- Python version: 3.8.5
- Operating System: macOS 10.15.4
- Install method (conda, pip, source): miniconda env, installed via pip
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with gcsfs/core.py at _open and GCSFile, then compare fsspec/spec.py where self.details is populated. Reproduce the supplied asynchronous example and determine the expected await behavior for fs.open and fp.read; done means the example no longer raises the coroutine or unclosed-session errors and the usage is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100