Better support for cloud authentication via tokens, service accounts, etc.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 59
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
In writing up the blog post for pins 1.1.0 in R, I ran into some challenges around authenticating for GCS in Python. I have a service account JSON file google-pins.json for a project "pins-dev" in my working directory.
I kind of expected that this might work, given what the docs for gcsfs and pins say (use a cached gcsfs token), but it does not:
import pins
import gcsfs
fs = gcsfs.GCSFileSystem(project="pins-dev", token="google-pins.json")
board = pins.board_gcs("pins-testing")
board.pin_read("small-numbers")
The pins functions don't work, even though fs.ls("pins-testing/") does.
This does work:
import pins
opts = {"cache_timeout": 0, "token": "google-pins.json"}
path = "pins-testing"
board = pins.board("gcs", path, storage_options=opts)
board.pin_read("nice-numbers")
Once I successfully read the pin this way, I can re-declare the board via pins.board_gcs("pins-testing") and still read the pin (which is cached locally), even though the board object is different.
🎯 Can/should we add a token argument to the GCS board? Should we do something similar for the other cloud boards? FWIW in R, we decided authentication was specific enough to these platforms that we needed to add individualized support in each board.
Also FWIW with GCS specifically, I'm still fuzzy on how the CLI authentication interacts with what I can do from Python. I did try authenticating via the CLI with gcloud auth application-default login and I'm not sure whether that was important.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the pins.board_gcs and pins.board entry points, comparing them with gcsfs.GCSFileSystem and the working storage_options example. Determine the intended token behavior for the GCS board and whether equivalent authentication support is needed for other cloud boards. Done means authenticated pins.board_gcs usage works without relying on a prior cached read, with the scope documented or tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, python
- Domain
- authentication, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100