googleapis / googleapis/google-cloud-java

[java-bigquerystorage] BigQueryWriteClient fails with UNAUTHENTICATED exception when private-service-connect endpoint is configured

オープン
#11,972 コメント 8 件 リアクション 4 件 担当者 0 名 GitHub で見る
api: bigquerystorage priority: p3 type: bug
主要言語
Java
スター
2.1k
フォーク
1.2k
平均マージ
1日 23時間
マージ済み PR(30日)
154

説明

#### Environment
* Using bigquerystorage version 2.34.2
* Using openjdk version 1.8.0_362
* Application is running on a custom GCP Compute Engine VM instance

#### Steps to reproduce
1. Create a private service connect endpoint in GCP (https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#create-endpoint)
2. Verify if the endpoint created above is reachable from the client vm
3. Create a BigQueryWriteClient object specifying the valid credentials and custom endpoint ( endpoint created in step 1 )
4. Create a JSONStreamWriter object using the writeClient created in step 3
5. Append data using the jsonStreamWriter
6. Append operation fails with UNAUTHENTICATED exception

#### Code example

```java
// The default endpoint used by BigQueryWriteClient is "bigquerystorage.googleapis.com:443"
// A private service connect endpoint is created with the endpoint name "samplepscendpoint"
// Hence, the private endpoint for bigquerystorage will be "bigquerystorage-samplepscendpoint.p.googleapis.com:443"
// Refer : https://cloud.google.com/vpc/docs/about-accessing-google-apis-endpoints
// nslookup on bigquerystorage-samplepscendpoint.p.googleapis.com is returning the correct internal ip as configured
String customEndpoint = "bigquerystorage-samplepscendpoint.p.googleapis.com:443";
FileInputStream serviceAccountStream = new FileInputStream(serviceAccountFilePath);
Credentials credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
BigQueryWriteSettings bigQueryWriteSettings = BigQueryWriteSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(credentials))
.setEndpoint(customEndpoint)
.build();
BigQueryWriteClient bigQueryWriteClient = BigQueryWriteClient.create(bigQueryWriteSettings);
streamWriter = JsonStreamWriter.newBuilder(tableName, client).build();
// build a jsonArray with some data
ApiFuture future = streamWriter.append(jsonArray);
// Append fails with UNAUTHENTICATED exception
// When customEndpoint is not explicitly set, data is appended as expected

// The same custom endpoint logic is working fine with BigQuery LegacyStreaming
// The default endpoint for BigQuery legacy streaming is "https://bigquery.googleapis.com"
String customHost = "https://bigquery-samplepscendpoint.p.googleapis.com";
BigQueryOptions bigqueryOptions = BigQueryOptions.newBuilder()
.setCredentials(credentials)
.setProjectId(projectId)
.setHost(customHost)
.build();
BigQuery client = bigqueryOptions.getService();
// build InsertAllRequest with data
InsertAllResponse response = client.insertAll(insertAllRequest);
// insertAll uploads data as expected
```

#### Stack trace
```
Exception in thread "main" com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:67)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:574)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:544)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:541)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:576)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:757)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:736)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.getWriteStream(BigQueryWriteClient.java:441)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.(JsonStreamWriter.java:472)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.(JsonStreamWriter.java:418)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.newBuilder(JsonStreamWriter.java:395)
at example.WriteApiDefaultMode$DataWriter.initialize(WriteApiDefaultMode.java:135)
at example.WriteApiDefaultMode.writeToDefaultStream(WriteApiDefaultMode.java:45)
at example.Main.writeWithDefaultMode(Main.java:60)
at example.Main.main(Main.java:39)
Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at io.grpc.Status.asRuntimeException(Status.java:539)
... 17 more
```

コントリビューションガイド

コントリビューションガイドを開く

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。