googleapis / googleapis/google-cloud-java
[java-storage] Excessive ALTS handshake logs from gke-metadata-server when using Java GCS client with gRPC
- 主要言語
- Java
- スター
- 2.1k
- フォーク
- 1.2k
- 平均マージ
- 1日 23時間
- マージ済み PR(30日)
- 154
説明
### Description
We are observing an extremely high volume of informational logs (approximately 500,000 to 1,000,000 per day) from the `gke-metadata-server` container on our GKE cluster.
This appears to be triggered by our Java application that uses the `google-cloud-storage` library with the gRPC transport option to interact with Google Cloud Storage. Our application handles a high volume of requests, writing to GCS approximately **1 million times per day, with peaks of up to 1,000 writes per second**.
The excessive logging is causing a significant increase in costs for our monitoring and logging tool (Datadog).
The most frequent log messages are related to ALTS session handshakes:
```
[rpc-id:b0b2453b42e5f6b2] Closing ALTS session from pod "ml-test-common-historical-1".
[rpc-id:b0b2453b42e5f6b2] Use node identity as channel identity to connect to service: "storage.googleapis.com"
[rpc-id:b0b2453b42e5f6b2] Starting ALTS session from pod "ml-test-common-historical-1" running as test-common/ml-test-common-app to act as ml-test-common@evaluation-mila.iam.gserviceaccount.com
[rpc-id:b0b2453b42e5f6b2] Processing ALTS handshake request from "10.80.12.18:53498".
```
Given the high traffic, we suspect that the client is creating new ALTS sessions too frequently, rather than reusing existing ones, which results in this verbose logging.
### Environment
* **`google-cloud-storage` version:** `2.55.0`
* **Transport:** gRPC (`StorageOptions.grpc()`)
* **Environment:** Google Kubernetes Engine (GKE)
* **GKE Cluster Version:** `1.33.4-gke.1036000`
* **Workload Identity Enabled:** Yes
* **Java Version:** `24`
### Code Snippet
Our `Storage` client is initialized as a singleton and reused throughout the application's lifecycle. The following code demonstrates our usage pattern.
```java
// This class is a singleton
class MyClass {
final Storage storage;
public MyClass() {
// Client is initialized once using gRPC transport
this.storage = StorageOptions.grpc()
.setProjectId("your-gcp-project-id")
.build()
.getService();
}
@Override
public void saveData(String dataId, ByteBuffer data) throws Exception {
BlobInfo blobInfo = toBlobInfo(dataId);
Blob blob = this.storage.create(blobInfo, BlobTargetOption.doesNotExist());
try (WriteChannel wc = blob.writer()){
wc.write(data);
}
}
}
```
### Investigation So Far
We opened a support case with the Google Kubernetes Engine team. Their recommendation was to **"adjust the client library's connection pool to prevent frequent session creation."**
However, it is not clear from the `google-cloud-storage` or `google-api-grpc` documentation how to configure or tune the underlying gRPC connection pool for this purpose.
コントリビューションガイド
評価
この issue はまだ評価されていません。