firebase / firebase/firebase-tools

Emulator increasing used memory and lose the connection with the Firestore

Open
#4,953 2 comments 0 reactions 0 assignees View on GitHub
emulators: firestore type: bug
Dominant language
TypeScript
Stars
4.5k
Forks
1.3k
Avg merge
1d 12h
Merged PRs (30d)
84

Description

### [REQUIRED] Environment info

**firebase-tools:** 11.7.0

**Platform:** Ubuntu 20.04.4 LTS (5.4.0-124-generic #140-Ubuntu SMP Thu Aug 4 02:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux)

**JVM**: Java(TM) SE Runtime Environment (build 17.0.4+11-LTS-179)

**Computer**: Acer NITRO5, i7, 16GB RAM, SSD 256GB.

### [REQUIRED] Test case

When I use a database exported from the production (the files exported from the production have 652MB). I import this backup when Firebase Emulator is starting with the parameter `--import`, when I am retrieving all documents (about 200.000) from a collection, after some seconds show the error message `Error: 14 UNAVAILABLE: No connection established` and the Emulator receives the signal to close `Firestore Emulator has exited upon receiving signal: SIGKILL`.

### [REQUIRED] Steps to reproduce

- Change the Java memory as the Firebase documentation explain https://firebase.google.com/docs/emulator-suite/install_and_configure#specifying_java_options.

> export JAVA_TOOL_OPTIONS="-Xmx8g -Xms8g"

- Start the Emulator

> firebase emulators:start --project demo-backup --import ./backup

- Run the code to retrieve the documents

> const path = '/collection';
> const account = db.collection(path);
>
> let count = 0;
>
> account.stream().on('data', (documentSnapshot) => {
> console.log(documentSnapshot.ref.path);
> ++count;
> }).on('end', () => {
> console.log(`Total count is ${count}`);
> });

### [REQUIRED] Expected behavior

I expect the code runs until shows the total.

### [REQUIRED] Actual behavior

**Error in the terminal**

> node:events:491
> throw er; // Unhandled 'error' event
> ^
>
> Error: 14 UNAVAILABLE: No connection established
> at Object.callErrorFromStatus (.../node_modules/@grpc/grpc-js/build/src/call.js:31:19)
> at Object.onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client.js:352:49)
> at Object.onReceiveStatus (.../node_modules/@grpc/grpc-js/build/src/client-interceptors.js:328:181)
> at .../node_modules/@grpc/grpc-js/build/src/call-stream.js:188:78
> at processTicksAndRejections (node:internal/process/task_queues:78:11)
> for call at
> at ServiceClientImpl.makeServerStreamRequest (.../node_modules/@grpc/grpc-js/build/src/client.js:336:30)
> at ServiceClientImpl. (.../node_modules/@grpc/grpc-js/build/src/make-client.js:105:19)
> at .../node_modules/firebase-admin/node_modules/@google-cloud/firestore/build/src/v1/firestore_client.js:206:29
> at .../node_modules/firebase-admin/node_modules/google-gax/build/src/streamingCalls/streamingApiCaller.js:38:28
> at .../node_modules/firebase-admin/node_modules/google-gax/build/src/normalCalls/timeout.js:44:16
> at Object.request (.../node_modules/firebase-admin/node_modules/google-gax/build/src/streamingCalls/streaming.js:126:40)
> at Timeout.makeRequest [as _onTimeout] (.../node_modules/firebase-admin/node_modules/google-gax/node_modules/retry-request/index.js:139:28)
> at listOnTimeout (node:internal/timers:559:17)
> at processTimers (node:internal/timers:502:7)
> Emitted 'error' event on Transform instance at:
> at Transform.onerror (node:internal/streams/readable:773:14)
> at Transform.emit (node:events:513:28)
> at emitErrorNT (node:internal/streams/destroy:157:8)
> at emitErrorCloseNT (node:internal/streams/destroy:122:3)
> at processTicksAndRejections (node:internal/process/task_queues:83:21) {
> code: 14,
> details: 'No connection established',
> metadata: Metadata { internalRepr: Map(0) {}, options: {} }
> }
>

**Debug log**

> [debug] [2022-08-23T00:39:15.888Z] ----------------------------------------------------------------------
> [debug] [2022-08-23T00:39:15.889Z] Command: /home/.../.nvm/versions/node/v16.17.0/bin/node /home/.../.nvm/versions/node/v16.17.0/bin/firebase emulators:start --project demo-backup --import ./backup
> [debug] [2022-08-23T00:39:15.890Z] CLI Version: 11.7.0
> [debug] [2022-08-23T00:39:15.890Z] Platform: linux
> [debug] [2022-08-23T00:39:15.890Z] Node Version: v16.17.0
> [debug] [2022-08-23T00:39:15.899Z] Time: Tue Aug 23 2022 12:39:15 GMT+1200 (New Zealand Standard Time)
> [debug] [2022-08-23T00:39:15.900Z] ----------------------------------------------------------------------
> [debug]
> [debug] [2022-08-23T00:39:15.974Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
> [debug] [2022-08-23T00:39:15.974Z] > authorizing via signed-in user (xxxx@xxxx)
> [debug] [2022-08-23T00:39:16.033Z] Picked up JAVA_TOOL_OPTIONS: -Xmx8g -Xms8g
>
> [debug] [2022-08-23T00:39:16.253Z] java version "17.0.4" 2022-07-19 LTS
>
> [debug] [2022-08-23T00:39:16.253Z] Java(TM) SE Runtime Environment (build 17.0.4+11-LTS-179)
> Java HotSpot(TM) 64-Bit Server VM (build 17.0.4+11-LTS-179, mixed mode, sharing)
>
> [debug] [2022-08-23T00:39:16.267Z] Parsed Java major version: 17
> [info] i emulators: Starting emulators: firestore {"metadata":{"emulator":{"name":"hub"},"message":"Starting emulators: firestore"}}
> [info] i emulators: Detected demo project ID "demo-backup", emulated services will use a demo configuration and attempts to access non-emulated services for this project will fail. {"metadata":{"emulator":{"name":"hub"},"message":"Detected demo project ID \"demo-backup\", emulated services will use a demo configuration and attempts to access non-emulated services for this project will fail."}}
> [debug] [2022-08-23T00:39:16.289Z] [hub] writing locator at /tmp/hub-demo-backup.json
> [info] i firestore: Detected non-emulator Firestore export at /home/dev/.../backup {"metadata":{"emulator":{"name":"firestore"},"message":"Detected non-emulator Firestore export at /home/dev/.../backup"}}
> [info] i firestore: Importing data from /home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata {"metadata":{"emulator":{"name":"firestore"},"message":"Importing data from /home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata"}}
> [warn] ⚠ firestore: Did not find a Cloud Firestore rules file specified in a firebase.json config file. {"metadata":{"emulator":{"name":"firestore"},"message":"Did not find a Cloud Firestore rules file specified in a firebase.json config file."}}
> [warn] ⚠ firestore: The emulator will default to allowing all reads and writes. Learn more about this option: https://firebase.google.com/docs/emulator-suite/install_and_configure#security_rules_configuration. {"metadata":{"emulator":{"name":"firestore"},"message":"The emulator will default to allowing all reads and writes. Learn more about this option: https://firebase.google.com/docs/emulator-suite/install_and_configure#security_rules_configuration."}}
> [debug] [2022-08-23T00:39:16.309Z] Ignoring unsupported arg: projectId {"metadata":{"emulator":{"name":"firestore"},"message":"Ignoring unsupported arg: projectId"}}
> [debug] [2022-08-23T00:39:16.309Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"firestore"},"message":"Ignoring unsupported arg: auto_download"}}
> [debug] [2022-08-23T00:39:16.309Z] Starting Firestore Emulator with command {"binary":"java","args":["-Dgoogle.cloud_firestore.debug_log_level=FINE","-Duser.language=en","-jar","/home/.../.cache/firebase/emulators/cloud-firestore-emulator-v1.14.4.jar","--host","localhost","--port",8180,"--seed_from_export","/home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata"],"optionalArgs":["port","webchannel_port","host","rules","functions_emulator","seed_from_export"],"joinArgs":false} {"metadata":{"emulator":{"name":"firestore"},"message":"Starting Firestore Emulator with command {\"binary\":\"java\",\"args\":[\"-Dgoogle.cloud_firestore.debug_log_level=FINE\",\"-Duser.language=en\",\"-jar\",\"/home/.../.cache/firebase/emulators/cloud-firestore-emulator-v1.14.4.jar\",\"--host\",\"localhost\",\"--port\",8180,\"--seed_from_export\",\"/home/dev/.../backup/2022-08-11T08:00:04_26595.overall_export_metadata\"],\"optionalArgs\":[\"port\",\"webchannel_port\",\"host\",\"rules\",\"functions_emulator\",\"seed_from_export\"],\"joinArgs\":false}"}}
> [info] i firestore: Firestore Emulator logging to firestore-debug.log {"metadata":{"emulator":{"name":"firestore"},"message":"Firestore Emulator logging to \u001b[1mfirestore-debug.log\u001b[22m"}}
> [debug] [2022-08-23T00:39:16.319Z] Picked up JAVA_TOOL_OPTIONS: -Xmx8g -Xms8g
> {"metadata":{"emulator":{"name":"firestore"},"message":"Picked up JAVA_TOOL_OPTIONS: -Xmx8g -Xms8g\n"}}
> [debug] [2022-08-23T00:39:23.451Z] Aug 23, 2022 12:39:23 PM com.google.cloud.datastore.emulator.firestore.websocket.WebSocketServer start
> INFO: Started WebSocket server on ws://localhost:41181
> {"metadata":{"emulator":{"name":"firestore"},"message":"Aug 23, 2022 12:39:23 PM com.google.cloud.datastore.emulator.firestore.websocket.WebSocketServer start\nINFO: Started WebSocket server on ws://localhost:41181\n"}}
> [debug] [2022-08-23T00:39:23.491Z] API endpoint: http://localhost:8180
> {"metadata":{"emulator":{"name":"firestore"},"message":"API endpoint: http://localhost:8180\n"}}
> [debug] [2022-08-23T00:39:23.491Z] If you are using a library that supports the FIRESTORE_EMULATOR_HOST environment variable, run:
>
> export FIRESTORE_EMULATOR_HOST=localhost:8180
>
> Dev App Server is now running.
>
> {"metadata":{"emulator":{"name":"firestore"},"message":"If you are using a library that supports the FIRESTORE_EMULATOR_HOST environment variable, run:\n\n export FIRESTORE_EMULATOR_HOST=localhost:8180\n\nDev App Server is now running.\n\n"}}
> [debug] [2022-08-23T00:39:23.616Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"ui"},"message":"Ignoring unsupported arg: auto_download"}}
> [debug] [2022-08-23T00:39:23.616Z] Ignoring unsupported arg: port {"metadata":{"emulator":{"name":"ui"},"message":"Ignoring unsupported arg: port"}}
> [debug] [2022-08-23T00:39:23.617Z] Starting Emulator UI with command {"binary":"node","args":["/home/.../.cache/firebase/emulators/ui-v1.9.0/server/server.js"],"optionalArgs":[],"joinArgs":false} {"metadata":{"emulator":{"name":"ui"},"message":"Starting Emulator UI with command {\"binary\":\"node\",\"args\":[\"/home/.../.cache/firebase/emulators/ui-v1.9.0/server/server.js\"],\"optionalArgs\":[],\"joinArgs\":false}"}}
> [info] i ui: Emulator UI logging to ui-debug.log {"metadata":{"emulator":{"name":"ui"},"message":"Emulator UI logging to \u001b[1mui-debug.log\u001b[22m"}}
> [debug] [2022-08-23T00:39:23.756Z] Web / API server started at localhost:4000
> {"metadata":{"emulator":{"name":"ui"},"message":"Web / API server started at localhost:4000\n"}}
> [info]
> ┌─────────────────────────────────────────────────────────────┐
> │ ✔ All emulators ready! It is now safe to connect your app. │
> │ i View Emulator UI at http://localhost:4000 │
> └─────────────────────────────────────────────────────────────┘
>
> ┌───────────┬────────────────┬─────────────────────────────────┐
> │ Emulator │ Host:Port │ View in Emulator UI │
> ├───────────┼────────────────┼─────────────────────────────────┤
> │ Firestore │ localhost:8180 │ http://localhost:4000/firestore │
> └───────────┴────────────────┴─────────────────────────────────┘
> Emulator Hub running at localhost:4400
> Other reserved ports: 4500
>
> Issues? Report them at https://github.com/firebase/firebase-tools/issues and attach the *-debug.log files.
>
> [debug] [2022-08-23T00:40:51.634Z] Aug 23, 2022 12:40:51 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
> INFO: Detected HTTP/2 connection.
> {"metadata":{"emulator":{"name":"firestore"},"message":"Aug 23, 2022 12:40:51 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead\nINFO: Detected HTTP/2 connection.\n"}}
> [debug] [2022-08-23T00:44:39.319Z] Aug 23, 2022 12:44:39 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
> INFO: Detected HTTP/2 connection.
> {"metadata":{"emulator":{"name":"firestore"},"message":"Aug 23, 2022 12:44:39 PM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead\nINFO: Detected HTTP/2 connection.\n"}}
> [warn] ⚠ Firestore Emulator has exited upon receiving signal: SIGKILL
>

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.