googleapis / googleapis/google-cloud-node

Large request payload crashing API

Open
#7,217 10 comments 1 reaction 1 assignee Claimed by @cindy-peng View on GitHub
api: logging library: logging-bunyan priority: p3 type: bug
Dominant language
TypeScript
Stars
3.2k
Forks
712
Avg merge
2d 3h
Merged PRs (30d)
99

Description

I am having the exact same issue described here: https://github.com/googleapis/nodejs-logging-bunyan/issues/14

Throughout the codebase, occasionally (and accidentally) I am logging very large entries, and encountering an exception which crashes the server:

```tsx
Error: 3 INVALID_ARGUMENT: Request payload size exceeds the limit: 10485760 bytes.
File "/workspace/node_modules/@grpc/grpc-js/build/src/call.js", line 31, col 19, in callErrorFromStatus
const error = new Error(message);
File "/workspace/node_modules/@grpc/grpc-js/build/src/client.js", line 192, col 76, in Object.onReceiveStatus
callProperties.callback((0, call_1.callErrorFromStatus)(status, /*callerStack*/ ''));
File "/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js", line 360, col 141, in Object.onReceiveStatus
'{snip} istener === null || listener === void 0 ? void 0 : listener.onReceiveStatus) === null || _b === void 0 ? void 0 : _b.call(listener, status);
File "/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js", line 323, col 181, in Object.onReceiveStatus
'{snip} ener === void 0 ? void 0 : interceptingListener.onReceiveStatus) === null || _b === void 0 ? void 0 : _b.call(interceptingListener, status);
File "/workspace/node_modules/@grpc/grpc-js/build/src/resolving-call.js", line 94, col 78, in
(_a = this.listener) === null || _a === void 0 ? void 0 : _a.onReceiveStatus(filteredStatus);
File "node:internal/process/task_queues", line 77, col 11, in process.processTicksAndRejections
```

I am using the logging-bunyan Express middleware as follows:

```tsx
const { logger: gcpLogger, mw } = await lb.express.middleware({
logName: "api-" + process.env.FLY_APP_NAME,
level: "debug",
defaultCallback: (err: any) => {
try {
if (err) {
console.error("Error occured with logger.", err)
Sentry.captureMessage(err, { level: "error" })
}
} catch (_err) {
console.error("Error occured with logger in try/catch blog.", _err)
Sentry.captureException(_err)
}
return true
},

// GCP should truncate the message down to 200kb.
// See https://github.com/googleapis/nodejs-logging/pull/607.
maxEntrySize: 200000,
})

```

It doesn't look like I'm able to pass in the arguments to this middleware as per the solution described here: https://github.com/googleapis/google-cloud-node-core/issues/480

maxEntrySize also doesn't seem to properly truncate.

Ideally, I don't care as much about truncating; I just want to properly handle the exception and not crash the server whenever I inadvertently attempt to log a massive entry.

#### Environment details

- OS: Linux
- Node.js version: 18.18.2
- npm version: 10.3.0
- `@google-cloud/logging-bunyan` version: 4.2.2

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.