googleapis / googleapis/google-cloud-ruby

[Storage] Easily generate signed URLs when using workload identity

Open
#13,307 4 comments 4 reactions 0 assignees View on GitHub
api: storage type: feature request
Dominant language
Ruby
Stars
1.4k
Forks
578
Avg merge
1d 11h
Merged PRs (30d)
166

Description

Hi, I am open to writing a PR to make code changes for this feature but wanted to start with an issue to get feedback.

When developing locally I can generate signed URL simply with:

```
require "google/cloud/storage"
storage = Google::Cloud::Storage.new(
project_id: "",
credentials: ENV["GCP_CREDENTIALS_PATH"]
)
bucket = storage.bucket ""
remote_file = bucket.file "foo/bar", skip_lookup: true
url = remote_file.signed_url method: "PUT", content_type: "application/json"
```

However, when running my code from within an eks pod with access grant via workload identitiy (https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity)

I get the error:
```
/usr/local/bundle/gems/google-cloud-storage-1.31.1/lib/google/cloud/storage/file/signer_v2.rb:88:in `determine_issuer': Service account credentials 'issuer (client_email)' is missing. To generate service account credentials see https://cloud.google.com/iam/docs/service-accounts (Google::Cloud::Storage::SignedUrlUnavailable)
```

Based on https://github.com/salrashid123/gcpsamples/blob/master/gcs_keyless_signedurl/main.py I was able to write some equivalent ruby code that works for generating a working signed URL (not the most well-factored code, just a proof-of-concept)
```
require "google/cloud/storage"
storage = Google::Cloud::Storage.new
bucket = storage.bucket ""
remote_file = bucket.file "foo/bar", skip_lookup: true
path = "/#{bucket.name}/#{remote_file.name}"
url_manually = "https://storage.googleapis.com#{path}"
access_token = JSON.parse(`curl -s -H 'Metadata-Flavor: Google' http://metadata/computeMetadata/v1/instance/service-accounts/default/token`)["access_token"]
connection = Faraday.new(url: "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/#{service_account_email}:signBlob") do |conn|
conn.request :json
end
expiration = 1.day.from_now.to_i
verb = "PUT"
content_md5 = ""
content_type = "application/json"
signature_string = "#{verb}\n#{content_md5}\n#{content_type}\n#{expiration}\n#{path}"
response = connection.post do |req|
req.headers["Authorization"] = "Bearer #{access_token}"
req.body = {delegates: [], payload: Base64.encode64(signature_string).gsub("\n","")}
end
signed_blob = JSON.parse(response.body)["signedBlob"]
signed_url = url_manually + "?" + URI.encode_www_form("GoogleAccessId": service_account_email, "Expires": expiration, "Signature": signed_blob)
```

So... I was thinking perhaps we could/should just change the implementation of `signed_url` to detect the conditions under which key-based signing will fail and automatically attempt this meta data service token + signBlob service method.

Or perhaps it's already exposed somewhere else within the depths of this ruby gem? (in which case in the very least the error message you see could be updated to point that out)

Thanks,
Jacob

---

Update based on https://googleapis.dev/ruby/google-cloud-storage/latest/Google/Cloud/Storage/File.html#signed_url-instance_method I have a better version

```
require "google/cloud/storage"
storage = Google::Cloud::Storage.new
bucket = storage.bucket ""
remote_file = bucket.file "foo/bar", skip_lookup: true
issuer = ""
signer = lambda do |string_to_sign|
IAMCredentials = Google::Apis::IamcredentialsV1
iam_client = IAMCredentials::IAMCredentialsService.new
scopes = ["https://www.googleapis.com/auth/iam"]
iam_client.authorization = Google::Auth.get_application_default scopes
request = Google::Apis::IamcredentialsV1::SignBlobRequest.new(
payload: string_to_sign
)
resource = "projects/-/serviceAccounts/#{issuer}"
response = iam_client.sign_service_account_blob resource, request
response.signed_blob
end
signed_url = remote_file.signed_url method: "PUT", content_type: "application/json", issuer: issuer, signer: signer
```

And the only things that are "custom" to my use case in this example end up being arguments into the `signed_url` method. so perhaps a simplification this gem could make would be to auto-generate the `signer` proc

Finally, I need a way to tell which version of the `signed_url` method args to use.

When running locally I have a credentials JSON file. so I need only call `remote_file.signed_url method: "PUT", content_type: "application/json"`
Whereas from my pod I have to call `remote_file.signed_url method: "PUT", content_type: "application/json", issuer: issuer, signer: signer`

If I do it backwards (specify an issue and signer when not on a pod) I get
```
/Users/jacob/.gem/ruby/3.0.1/gems/googleauth-0.16.2/lib/googleauth/application_default.rb:76:in `get_application_default': Could not load the default credentials. Browse to (RuntimeError)
https://developers.google.com/accounts/docs/application-default-credentials
for more information
```

Is there some simple call I could make to this gem to determine what context I'm in?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.