googleapis / googleapis/google-cloud-java

[java-bigquerystorage] BigQueryWriteClient fails with UNAUTHENTICATED exception when private-service-connect endpoint is configured

Đang mở
#11,972 8 bình luận 4 reaction 0 người được giao Xem trên GitHub
api: bigquerystorage priority: p3 type: bug
Ngôn ngữ chính
Java
Star
2.1k
Fork
1.2k
Merge trung bình
1 ngày 23 giờ
Pull request đã merge (30 ngày)
154

Mô tả

#### Environment
* Using bigquerystorage version 2.34.2
* Using openjdk version 1.8.0_362
* Application is running on a custom GCP Compute Engine VM instance

#### Steps to reproduce
1. Create a private service connect endpoint in GCP (https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#create-endpoint)
2. Verify if the endpoint created above is reachable from the client vm
3. Create a BigQueryWriteClient object specifying the valid credentials and custom endpoint ( endpoint created in step 1 )
4. Create a JSONStreamWriter object using the writeClient created in step 3
5. Append data using the jsonStreamWriter
6. Append operation fails with UNAUTHENTICATED exception

#### Code example

```java
// The default endpoint used by BigQueryWriteClient is "bigquerystorage.googleapis.com:443"
// A private service connect endpoint is created with the endpoint name "samplepscendpoint"
// Hence, the private endpoint for bigquerystorage will be "bigquerystorage-samplepscendpoint.p.googleapis.com:443"
// Refer : https://cloud.google.com/vpc/docs/about-accessing-google-apis-endpoints
// nslookup on bigquerystorage-samplepscendpoint.p.googleapis.com is returning the correct internal ip as configured
String customEndpoint = "bigquerystorage-samplepscendpoint.p.googleapis.com:443";
FileInputStream serviceAccountStream = new FileInputStream(serviceAccountFilePath);
Credentials credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
BigQueryWriteSettings bigQueryWriteSettings = BigQueryWriteSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(credentials))
.setEndpoint(customEndpoint)
.build();
BigQueryWriteClient bigQueryWriteClient = BigQueryWriteClient.create(bigQueryWriteSettings);
streamWriter = JsonStreamWriter.newBuilder(tableName, client).build();
// build a jsonArray with some data
ApiFuture future = streamWriter.append(jsonArray);
// Append fails with UNAUTHENTICATED exception
// When customEndpoint is not explicitly set, data is appended as expected

// The same custom endpoint logic is working fine with BigQuery LegacyStreaming
// The default endpoint for BigQuery legacy streaming is "https://bigquery.googleapis.com"
String customHost = "https://bigquery-samplepscendpoint.p.googleapis.com";
BigQueryOptions bigqueryOptions = BigQueryOptions.newBuilder()
.setCredentials(credentials)
.setProjectId(projectId)
.setHost(customHost)
.build();
BigQuery client = bigqueryOptions.getService();
// build InsertAllRequest with data
InsertAllResponse response = client.insertAll(insertAllRequest);
// insertAll uploads data as expected
```

#### Stack trace
```
Exception in thread "main" com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:67)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:574)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:544)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:541)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:576)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:757)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:736)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.getWriteStream(BigQueryWriteClient.java:441)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.(JsonStreamWriter.java:472)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.(JsonStreamWriter.java:418)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.newBuilder(JsonStreamWriter.java:395)
at example.WriteApiDefaultMode$DataWriter.initialize(WriteApiDefaultMode.java:135)
at example.WriteApiDefaultMode.writeToDefaultStream(WriteApiDefaultMode.java:45)
at example.Main.writeWithDefaultMode(Main.java:60)
at example.Main.main(Main.java:39)
Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at io.grpc.Status.asRuntimeException(Status.java:539)
... 17 more
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.