method to track session stats
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
I couldn't find a method to keep track of the total data cached and transferred on a fsspec session but it would be really helpful to have something like that. Maybe there is already a way to calculate this? given that I do the following
s3 = s3fs.S3FileSystem(anon=True, data_tracking=True)
# I do the usual
s3.open(...)
# and then I can ask:
stats = s3.stats()
and stats will have total data transferred, total HTTP calls, current cache size, max cache size etc.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the S3FileSystem constructor with data_tracking=True and follow the s3.open(...) session flow. Define how stats() should report total data transferred, total HTTP calls, current cache size, and maximum cache size, then verify the returned stats against those documented expectations.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100