googleapis / googleapis/google-cloud-node

Cloud Secrets Manager often errors DEADLINE Exceeded

Open
#5,043 11 comments 5 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
3.2k
Forks
712
Avg merge
2d 3h
Merged PRs (30d)
99

Description

⏳ `Secrets Manager retrieved in 31.092 seconds`

So slow!

----------------------
Yes I have tried the quick fixes and google cloud docs & github/stackoverflow search.

#### Environment details

- which product (packages/*): @google-cloud/secret-manager
- OS: Linux Google Cloud Run
- Node.js version: v18
- npm version: 9.5
- google-cloud-node version: 5.0.1

#### Steps to reproduce

About 25% of the time, cloud secrets manager will timeout when attempting to access a secret. I have logged avg duration to get a secret and its around 10-30 seconds on first attempt (and <1 sec thereafter). The code is run within firebase functions and cloud run on us-central1. All my secrets managers are set to auto-replication policy.

What is causing this issue and how can I fix it? It's severely critical.

The pattern I am using:

```js
const secret = await getSecretFromGCloud() // this takes 10+ seconds
axios.post(url, payload, headers)
```

Error:

Error: 4 DEADLINE_EXCEEDED: Deadline exceeded
at callErrorFromStatus (/workspace/node_modules/@google-cloud/secret-manager/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/workspace/node_modules/@google-cloud/secret-manager/node_modules/@grpc/grpc-js/build/src/client.js:192:76)
at Object.onReceiveStatus (/workspace/node_modules/@google-cloud/secret-manager/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:360:141)
at Object.onReceiveStatus (/workspace/node_modules/@google-cloud/secret-manager/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
at /workspace/node_modules/@google-cloud/secret-manager/node_modules/@grpc/grpc-js/build/src/resolving-call.js:99:78
at processTicksAndRejections (node:internal/process/task_queues:77:11)
at runNextTicks (node:internal/process/task_queues:64:3)
at process.processTimers (node:internal/timers:509:9)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.