rstudio / rstudio/pins-python

Better support for cloud authentication via tokens, service accounts, etc.

Open
#181 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
59
Forks
11
PR merge metrics
No merged PRs in 30d

Description

In writing up the blog post for pins 1.1.0 in R, I ran into some challenges around authenticating for GCS in Python. I have a service account JSON file google-pins.json for a project "pins-dev" in my working directory.

I kind of expected that this might work, given what the docs for gcsfs and pins say (use a cached gcsfs token), but it does not:

import pins
import gcsfs

fs = gcsfs.GCSFileSystem(project="pins-dev", token="google-pins.json")
board = pins.board_gcs("pins-testing")
board.pin_read("small-numbers")

The pins functions don't work, even though fs.ls("pins-testing/") does.

This does work:

import pins
opts = {"cache_timeout": 0, "token": "google-pins.json"}
path = "pins-testing"
board = pins.board("gcs", path, storage_options=opts)
board.pin_read("nice-numbers")

Once I successfully read the pin this way, I can re-declare the board via pins.board_gcs("pins-testing") and still read the pin (which is cached locally), even though the board object is different.

🎯 Can/should we add a token argument to the GCS board? Should we do something similar for the other cloud boards? FWIW in R, we decided authentication was specific enough to these platforms that we needed to add individualized support in each board.

Also FWIW with GCS specifically, I'm still fuzzy on how the CLI authentication interacts with what I can do from Python. I did try authenticating via the CLI with gcloud auth application-default login and I'm not sure whether that was important.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the pins.board_gcs and pins.board entry points, comparing them with gcsfs.GCSFileSystem and the working storage_options example. Determine the intended token behavior for the GCS board and whether equivalent authentication support is needed for other cloud boards. Done means authenticated pins.board_gcs usage works without relying on a prior cached read, with the scope documented or tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
authentication, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.