firebase / firebase/firebase-tools

Cannot determine payload type, datacontenttype is undefined

Open
#9,035 8 comments 4 reactions 0 assignees View on GitHub
emulators: functions reproducible type: bug
Dominant language
TypeScript
Stars
4.5k
Forks
1.3k
Avg merge
1d 12h
Merged PRs (30d)
84

Description

### [REQUIRED] Environment info

**firebase-tools:**
14.14.0

**Platform:**
macOS (latest)

### [REQUIRED] Test case

I made a repro project https://github.com/EthanSK/onobjectfinalize-indirect

Detailed (but ai generated) intsructions are in the readme

It was hard to reliably get it to break at the same point every time, hence the multiple iterations to try and trigger the issue.

### [REQUIRED] Steps to reproduce


make sure to do this in the inner node_modules under functions/, not the project root node modules. sorry about that:

add this log to line 186 of firestore.js , at the start of createBeforeSnapshot in functions/node_modules/firebase-functions/lib/v2/providers/firestore.js

console.log("create before snapshot: ", event)

, put a debug breakpoint on this line

logger.error(`Cannot determine payload type, datacontenttype is ${event.datacontenttype}, failing out.`);

in one terminal, run `npm run serve` , then once thats booted up , in another terminal run `./test-race-conditions.sh`

make sure the debugger is running

**result:**
after some iterations, the breakpoint should be activated. If you inspect the event data, you'll see it's a storage event, but this method seems only equipped to handle firestore events. This causes the function to error out and break for the remaining function invocations, rendering local development useless.

This is a serious dev breaking bug as my whole function instance shuts down after the error occurs.

this only works with debugger / inspect-functions

i know this might seem like an extreme overload and spam of the poor emulator, but in my actual app the overload conditions are much less and it happens consistently, but i couldnt quite boil down exactly the right cause, so i included all the possible reasons in the repro, and after a lot of wrangling got it to finally work consistently enough.

### [REQUIRED] Expected behavior

It should not error, storage events should not be going through createBeforeSnapshot maybe? and not error out.

A hint of what might be happening may be to do with some race condition when storage triggers start overlapping, in a context that was iniated from firestore triggers. When the sleep between invocations is increased, this issue becomes harder and harde to repro.

### [REQUIRED] Actual behavior

Storage events are going through firestore's createBeforeSnapshot in the emulators (sometimes, it is not predictable when this happens but its very likely to happen after some iterations in the repro project)

This only seems to happen under more intense load that overwhelms the storage emulator? or some emulator i dont know exactly what it could be (although I inspected the source code and had extensive conversations with AI)

```
> }
> {"severity":"ERROR","message":"Error: Cannot determine payload type, datacontenttype is undefined, failing out.\n at entryFromArgs (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/logger/index.js:134:19)\n at Object.error (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/logger/index.js:121:11)\n at createBeforeSnapshot (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:194:16)\n at makeChangedFirestoreEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:236:37)\n at func (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:317:32)\n at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:524:16\n at runFunction (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:15)\n at runCloudEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:523:11)\n at processBackground (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:490:16)\n at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:690:27"}
⚠ functions: Error: Error: Cannot parse event payload.
at createBeforeSnapshot (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:195:15)
at makeChangedFirestoreEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:236:37)
at func (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/functions/node_modules/firebase-functions/lib/v2/providers/firestore.js:317:32)
at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:524:16
at runFunction (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:506:15)
at runCloudEvent (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:523:11)
at processBackground (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:490:16)
at /Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/firebase-tools/lib/emulator/functionsEmulatorRuntime.js:690:27
at Layer.handle [as handle_request] (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/express/lib/router/layer.js:95:5)
at next (/Users/ethansarif-kattan/Projects/aimvs repros/onobjectfinalize-indirect/node_modules/express/lib/router/route.js:149:13)
⚠ Your function was killed because it raised an unhandled error.
i functions: Finished "us-central1-onUploadFileFinalize" in 16603.434ms
```

Contributor guide

Open the contributing guide

Research direction

Start with functions/node_modules/firebase-functions/lib/v2/providers/firestore.js, especially createBeforeSnapshot and makeChangedFirestoreEvent, then inspect the functions emulator path in functionsEmulatorRuntime.js. Run the linked onobjectfinalize-indirect reproduction with npm run serve and ./test-race-conditions.sh under the debugger. Done means storage events no longer reach the Firestore payload parser or terminate remaining invocations.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
cli, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.