googleapis / googleapis/google-cloud-java

[java-bigquerystorage] BigQueryWriteClient fails with UNAUTHENTICATED exception when private-service-connect endpoint is configured

Abierto
#11,972 8 comentarios 4 reacciones 0 asignados Ver en GitHub
api: bigquerystorage priority: p3 type: bug
Lenguaje dominante
Java
Estrellas
2.1k
Forks
1.2k
Merge medio
1 d 23 h
PR fusionados (30 d)
154

Descripción

#### Environment
* Using bigquerystorage version 2.34.2
* Using openjdk version 1.8.0_362
* Application is running on a custom GCP Compute Engine VM instance

#### Steps to reproduce
1. Create a private service connect endpoint in GCP (https://cloud.google.com/vpc/docs/configure-private-service-connect-apis#create-endpoint)
2. Verify if the endpoint created above is reachable from the client vm
3. Create a BigQueryWriteClient object specifying the valid credentials and custom endpoint ( endpoint created in step 1 )
4. Create a JSONStreamWriter object using the writeClient created in step 3
5. Append data using the jsonStreamWriter
6. Append operation fails with UNAUTHENTICATED exception

#### Code example

```java
// The default endpoint used by BigQueryWriteClient is "bigquerystorage.googleapis.com:443"
// A private service connect endpoint is created with the endpoint name "samplepscendpoint"
// Hence, the private endpoint for bigquerystorage will be "bigquerystorage-samplepscendpoint.p.googleapis.com:443"
// Refer : https://cloud.google.com/vpc/docs/about-accessing-google-apis-endpoints
// nslookup on bigquerystorage-samplepscendpoint.p.googleapis.com is returning the correct internal ip as configured
String customEndpoint = "bigquerystorage-samplepscendpoint.p.googleapis.com:443";
FileInputStream serviceAccountStream = new FileInputStream(serviceAccountFilePath);
Credentials credentials = ServiceAccountCredentials.fromStream(serviceAccountStream);
BigQueryWriteSettings bigQueryWriteSettings = BigQueryWriteSettings.newBuilder()
.setCredentialsProvider(FixedCredentialsProvider.create(credentials))
.setEndpoint(customEndpoint)
.build();
BigQueryWriteClient bigQueryWriteClient = BigQueryWriteClient.create(bigQueryWriteSettings);
streamWriter = JsonStreamWriter.newBuilder(tableName, client).build();
// build a jsonArray with some data
ApiFuture future = streamWriter.append(jsonArray);
// Append fails with UNAUTHENTICATED exception
// When customEndpoint is not explicitly set, data is appended as expected

// The same custom endpoint logic is working fine with BigQuery LegacyStreaming
// The default endpoint for BigQuery legacy streaming is "https://bigquery.googleapis.com"
String customHost = "https://bigquery-samplepscendpoint.p.googleapis.com";
BigQueryOptions bigqueryOptions = BigQueryOptions.newBuilder()
.setCredentials(credentials)
.setProjectId(projectId)
.setHost(customHost)
.build();
BigQuery client = bigqueryOptions.getService();
// build InsertAllRequest with data
InsertAllResponse response = client.insertAll(insertAllRequest);
// insertAll uploads data as expected
```

#### Stack trace
```
Exception in thread "main" com.google.api.gax.rpc.UnauthenticatedException: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:116)
at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:67)
at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:574)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:544)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:541)
at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:576)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:757)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:736)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.getWriteStream(BigQueryWriteClient.java:441)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.(JsonStreamWriter.java:472)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter$Builder.(JsonStreamWriter.java:418)
at com.google.cloud.bigquery.storage.v1.JsonStreamWriter.newBuilder(JsonStreamWriter.java:395)
at example.WriteApiDefaultMode$DataWriter.initialize(WriteApiDefaultMode.java:135)
at example.WriteApiDefaultMode.writeToDefaultStream(WriteApiDefaultMode.java:45)
at example.Main.writeWithDefaultMode(Main.java:60)
at example.Main.main(Main.java:39)
Caused by: io.grpc.StatusRuntimeException: UNAUTHENTICATED: Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
at io.grpc.Status.asRuntimeException(Status.java:539)
... 17 more
```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.