firebase / firebase/firebase-tools

PubSub emulator consumes more and more CPU when background triggers are toggled

Open
#6,585 3 comments 0 reactions 1 assignee Claimed by @joehan View on GitHub
emulators: pubsub type: bug
Dominant language
TypeScript
Stars
4.5k
Forks
1.3k
Avg merge
1d 12h
Merged PRs (30d)
84

Description

Synopsis: When the PubSub emulator is running, and the "background triggers" (for Functions) are toggled off and on, the PubSub emulator starts to consume more and more CPU, until it is eventually consuming all available CPU.

### [REQUIRED] Environment info

**firebase-tools:** 13.0.0
Also tested with 12.4.7

**Platform:** MacOS

### [REQUIRED] Test case

Initial project setup:

```sh
firebase init functions # select Typescript
firebase init emulators # select PubSub, Functions, Firestore
```

Set up some functions, `functions/src/index.ts` looks like this:

```ts
import {onMessagePublished} from "firebase-functions/v2/pubsub";
import {onDocumentWritten} from "firebase-functions/v2/firestore";
import * as logger from "firebase-functions/logger";

export const helloPubsub = onMessagePublished("some-topic", () => {
logger.info("Hello pubsub!");
});

export const helloFirestore = onDocumentWritten("/some/doc", () => {
logger.info("Hello firestore!");
});
```

This is a minimal setup, but I first noticed this problem on a real, larger project. (And I think it's easier to cause to happen the more functions there are defined).

### [REQUIRED] Steps to reproduce

Start the emulators:

```sh
cd functions && npm run build && cd ..
firebase emulators:start
```

We're looking for _at least_ the PubSub and Functions emulators to be started. The setup described above will also start the Firestore emulator.

Now, we want to turn background triggers on and off repeatedly:

```sh
while true; do
curl -X PUT http://127.0.0.1:4400/functions/disableBackgroundTriggers;
sleep 1;
curl -X PUT http://127.0.0.1:4400/functions/enableBackgroundTriggers;
sleep 1;
done
```

Modify if you're using a different Emulator Hub address or port. Let this run for a while. The functions will be reinitializing with each run:

```
i emulators: Disabling Cloud Functions triggers, non-HTTP functions will not execute.
i functions[helloPubsub]: function temporarily disabled.
i functions[helloFirestore]: function temporarily disabled.
i emulators: Enabling Cloud Functions triggers, non-HTTP functions will execute.
✔ functions: Using node@18 from host.
Serving at port 8XXX

✔ functions: Loaded functions definitions from source: helloPubsub, helloFirestore.
✔ functions[us-central1-helloPubsub]: pubsub function initialized.
✔ functions[us-central1-helloFirestore]: firestore function initialized.
```

With only these two functions defined, you may need to let this script run for a few minutes (at least a hundred or two hundred toggles) before the actual behavior described below becomes very visible. But with many different PubSub functions defined (e.g. in a real project) the behavior is much more visible, because it happens with far fewer toggles.

### [REQUIRED] Expected behavior

Monitor the CPU usage of the PubSub emulator. We'd expect it to be using some small amount of CPU to reinitialize (because each reinitialization of the functions might be creating a topic?)

### [REQUIRED] Actual behavior

Monitoring the CPU usage of the PubSub emulator, what we actually see is it using more and more CPU, even if we pause or stop the script that is toggling the background triggers.

If you continue to toggle the background triggers, eventually the PubSub emulator will consume all available CPU on the machine. This can happen quite rapidly with many PubSub triggered functions defined.

This additional CPU usage never goes away e.g. if you completely stop the script that is toggling the background triggers, the CPU usage of the PubSub emulator does not ever go back down to idle levels of CPU usage. (e.g. it will continue to use 200% or 400%)

### Notes

- I'm toggling background triggers in my real project e.g. to prevent triggers while running functional tests against the emulators
- I suspect, but haven't proven yet, that the CPU usage goes up proportional to the number of PubSub topics defined in Functions

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.