firebase / firebase/firebase-tools

Dataconnect emulator fails to migrate data from Postgres 16 to 17

Open
#9,259 5 comments 0 reactions 0 assignees View on GitHub
emulator: dataconnect reproducible type: bug
Dominant language
TypeScript
Stars
4.5k
Forks
1.3k
Avg merge
1d 12h
Merged PRs (30d)
84

Description

### [REQUIRED] Environment info

**firebase-tools:** 14.18.0

**Platform:** macOS

### [REQUIRED] Test case

N/A

### [REQUIRED] Steps to reproduce

1. Export from emulator using `firebase-tools` version **14.7.0** or earlier with command `firebase emulators:start --import=./emulator-data --export-on-exit`.
2. Update `firebase-tools` to version **14.8.0** or later
3. Start emulator again using above command
4. See migration issue crash

### [REQUIRED] Expected behavior

The emulator to start smoothly using the migrated version of postgres 16

### [REQUIRED] Actual behavior

```
⚠ dataconnect: 'firebase.json#emulators.dataconnect.dataDir' is set and `--import` flag was passed. This will overwrite any data saved from previous runs.
✔ Do you wish to continue and overwrite data in dataconnect/.dataconnect/pgliteData? Yes
i dataconnect: Importing data from /usr/path/to/project/functions/emulator-data/dataconnect_export
⚠ dataconnect: Operations that use vector_embed will make calls to production Vertex AI
i dataconnect: Data Connect Emulator logging to dataconnect-debug.log
i dataconnect: Started up Postgres server, listening on {"address":"0.0.0.0","family":"IPv4","port":5432}
Detected a Postgres 16 data directory from an older version of firebase-tools. Migrating to Postgres 17...
Opening database with Postgres 16...
Error: Database already exists, cannot load from tarball
at mr.vs (/usr/local/lib/node_modules/firebase-tools/node_modules/pglite-2/dist/index.cjs:12:243301)
at async PostgresServer.migrateDb (/usr/local/lib/node_modules/firebase-tools/lib/emulator/dataconnect/pgliteServer.js:136:9)
at async PostgresServer.getDb (/usr/local/lib/node_modules/firebase-tools/lib/emulator/dataconnect/pgliteServer.js:107:23)
⬢ dataconnect: Postgres threw an unexpected error, shutting down the Data Connect emulator: AbortError: The operation was aborted
i emulators: Shutting down emulators.
```

Verbose

```
[warn] ⚠ dataconnect: 'firebase.json#emulators.dataconnect.dataDir' is set and `--import` flag was passed. This will overwrite any data saved from previous runs. {"metadata":{"emulator":{"name":"dataconnect"},"message":"'firebase.json#emulators.dataconnect.dataDir' is set and `--import` flag was passed. This will overwrite any data saved from previous runs."}}
[info] i dataconnect: Importing data from /usr/path/to/project/functions/emulator-data/dataconnect_export {"metadata":{"emulator":{"name":"dataconnect"},"message":"Importing data from /usr/path/to/project/functions/emulator-data/dataconnect_export"}}
[debug] [2025-10-06T08:28:03.523Z] defaultcredentials: writing to file /usr/.config/firebase/my_gmail_com_application_default_credentials.json
[debug] [2025-10-06T08:28:03.810Z] I1006 13:58:03.790585 16966 control.go:94] [/emulator/p-dataconnect 72ae] UpdateResources(): done
Main schema: sources: schema/schema.gql [1008B]
Connector "default": sources: connector/queries.gql [908B] connector/mutations.gql [890B]
{"metadata":{"emulator":{"name":"dataconnect"},"message":"I1006 13:58:03.790585 16966 control.go:94] [/emulator/p-dataconnect 72ae] UpdateResources(): done\nMain schema: sources: schema/schema.gql [1008B] \nConnector \"default\": sources: connector/queries.gql [908B] connector/mutations.gql [890B] \n"}}
[warn] ⚠ dataconnect: Operations that use vector_embed will make calls to production Vertex AI {"metadata":{"emulator":{"name":"dataconnect"},"message":"Operations that use vector_embed will make calls to production Vertex AI"}}
[debug] [2025-10-06T08:28:03.810Z] defaultcredentials: writing to file /usr/.config/firebase/my_gmail_com_application_default_credentials.json
[debug] [2025-10-06T08:28:03.811Z] Ignoring unsupported arg: auto_download {"metadata":{"emulator":{"name":"dataconnect"},"message":"Ignoring unsupported arg: auto_download"}}
[debug] [2025-10-06T08:28:03.811Z] Ignoring unsupported arg: port {"metadata":{"emulator":{"name":"dataconnect"},"message":"Ignoring unsupported arg: port"}}
[debug] [2025-10-06T08:28:03.811Z] Starting Data Connect Emulator with command {"binary":"/usr/.cache/firebase/emulators/dataconnect-emulator-2.14.0","args":["--logtostderr","-v=2","dev","--listen=0.0.0.0:9399","--config_dir=/usr/path/to/project/dataconnect","--enable_output_schema_extensions=true","--enable_output_generated_sdk=true"],"optionalArgs":["listen","config_dir","enable_output_schema_extensions","enable_output_generated_sdk"],"joinArgs":true,"shell":false,"port":9399} {"metadata":{"emulator":{"name":"dataconnect"},"message":"Starting Data Connect Emulator with command {\"binary\":\"/usr/.cache/firebase/emulators/dataconnect-emulator-2.14.0\",\"args\":[\"--logtostderr\",\"-v=2\",\"dev\",\"--listen=0.0.0.0:9399\",\"--config_dir=/usr/path/to/project/dataconnect\",\"--enable_output_schema_extensions=true\",\"--enable_output_generated_sdk=true\"],\"optionalArgs\":[\"listen\",\"config_dir\",\"enable_output_schema_extensions\",\"enable_output_generated_sdk\"],\"joinArgs\":true,\"shell\":false,\"port\":9399}"}}
[info] i dataconnect: Data Connect Emulator logging to dataconnect-debug.log {"metadata":{"emulator":{"name":"dataconnect"},"message":"Data Connect Emulator logging to \u001b[1mdataconnect-debug.log\u001b[22m"}}
[info] i dataconnect: Started up Postgres server, listening on {"address":"0.0.0.0","family":"IPv4","port":5432} {"metadata":{"emulator":{"name":"dataconnect"},"message":"Started up Postgres server, listening on {\"address\":\"0.0.0.0\",\"family\":\"IPv4\",\"port\":5432}"}}
[debug] [2025-10-06T08:28:03.825Z] [Data Connect] Connecting to postgres://127.0.0.1:5432/postgres?sslmode=disable}...
[debug] [2025-10-06T08:28:03.825Z] >>> [apiv2][query] POST http://127.0.0.1:9399/emulator/configure [none]
[debug] [2025-10-06T08:28:03.825Z] >>> [apiv2][body] POST http://127.0.0.1:9399/emulator/configure {"connectionString":"postgres://127.0.0.1:5432/postgres?sslmode=disable","database":"postgres","serviceId":"p-dataconnect","maxOpenConnections":1}
[debug] [2025-10-06T08:28:03.826Z] *** [apiv2] error from fetch(http://127.0.0.1:9399/emulator/configure, {"headers":{},"method":"POST","body":"{\"connectionString\":\"postgres://127.0.0.1:5432/postgres?sslmode=disable\",\"database\":\"postgres\",\"serviceId\":\"p-dataconnect\",\"maxOpenConnections\":1}"}): FetchError: request to http://127.0.0.1:9399/emulator/configure failed, reason: connect ECONNREFUSED 127.0.0.1:9399
[debug] [2025-10-06T08:28:03.826Z] [Data Connect] Retrying connectToPostgress call (1 of 3 attempts): FirebaseError: Data Connect emulator: undefined
[debug] [2025-10-06T08:28:03.984Z] I1006 13:58:03.983390 16967 load.go:31] Reloading schema and connectors...
{"metadata":{"emulator":{"name":"dataconnect"},"message":"I1006 13:58:03.983390 16967 load.go:31] Reloading schema and connectors...\n"}}
[debug] [2025-10-06T08:28:04.069Z] I1006 13:58:04.069366 16967 control.go:94] [/emulator/p-dataconnect f1e3] UpdateResources(): done
Main schema: sources: schema/schema.gql [1008B]
Connector "default": sources: connector/mutations.gql [890B] connector/queries.gql [908B]
{"metadata":{"emulator":{"name":"dataconnect"},"message":"I1006 13:58:04.069366 16967 control.go:94] [/emulator/p-dataconnect f1e3] UpdateResources(): done\nMain schema: sources: schema/schema.gql [1008B] \nConnector \"default\": sources: connector/mutations.gql [890B] connector/queries.gql [908B] \n"}}
[debug] [2025-10-06T08:28:04.098Z] I1006 13:58:04.098817 16967 collector.go:107] schema extensions wrote into "/usr/path/to/project/dataconnect/.dataconnect/schema"
Generated sources: main/mutation.gql [6678B] prelude.gql [73219B] main/implicit.gql [282B] main/input.gql [18199B] main/relation.gql [6181B] main/query.gql [4629B]
{"metadata":{"emulator":{"name":"dataconnect"},"message":"I1006 13:58:04.098817 16967 collector.go:107] schema extensions wrote into \"/usr/path/to/project/dataconnect/.dataconnect/schema\"\nGenerated sources: main/mutation.gql [6678B] prelude.gql [73219B] main/implicit.gql [282B] main/input.gql [18199B] main/relation.gql [6181B] main/query.gql [4629B] \n"}}
[debug] [2025-10-06T08:28:04.099Z] I1006 13:58:04.098985 16967 load.go:109] Finished reloading
{"metadata":{"emulator":{"name":"dataconnect"},"message":"I1006 13:58:04.098985 16967 load.go:109] Finished reloading\n"}}
[debug] [2025-10-06T08:28:04.102Z] I1006 13:58:04.102305 16967 dev.go:102] Listening on address (HTTP + gRPC): [::]:9399
{"metadata":{"emulator":{"name":"dataconnect"},"message":"I1006 13:58:04.102305 16967 dev.go:102] Listening on address (HTTP + gRPC): [::]:9399\n"}}
[debug] [2025-10-06T08:28:05.827Z] [Data Connect] Connecting to postgres://127.0.0.1:5432/postgres?sslmode=disable}...
[debug] [2025-10-06T08:28:05.832Z] >>> [apiv2][query] POST http://127.0.0.1:9399/emulator/configure [none]
[debug] [2025-10-06T08:28:05.833Z] >>> [apiv2][body] POST http://127.0.0.1:9399/emulator/configure {"connectionString":"postgres://127.0.0.1:5432/postgres?sslmode=disable","database":"postgres","serviceId":"p-dataconnect","maxOpenConnections":1}
[debug] [2025-10-06T08:28:05.893Z] Importing from /usr/path/to/project/functions/emulator-data/dataconnect_export/postgres.tar.gz
[debug] [2025-10-06T08:28:05.896Z] Found Postgres version file with version: 16
[info] Detected a Postgres 16 data directory from an older version of firebase-tools. Migrating to Postgres 17...
[info] Opening database with Postgres 16...
[error] ⬢ dataconnect: Postgres threw an unexpected error, shutting down the Data Connect emulator: AbortError: The operation was aborted {"metadata":{"emulator":{"name":"dataconnect"},"message":"Postgres threw an unexpected error, shutting down the Data Connect emulator: AbortError: The operation was aborted"}}
[info] i emulators: Shutting down emulators. {"metadata":{"emulator":{"name":"hub"},"message":"Shutting down emulators."}}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.