firebase / firebase/firebase-js-sdk

Connection fails and never recovers

Open
#6,537 13 comments 1 reaction 0 assignees View on GitHub
api: database needs-attention question
Dominant language
TypeScript
Stars
5.1k
Forks
1k
Avg merge
2d 21h
Merged PRs (30d)
37

Description

### [REQUIRED] Describe your environment

* Operating System version: MacOS 12.5
* Browser version: Chromium 100
* Firebase SDK version: 9.9
* Firebase Product: realtime database
*
### [REQUIRED] Describe the problem

When changing connections, it's possible to get firebase into a 'stuck' state where all requests return `datastale`, and transactions error out with error `maxretry`.

#### Steps to reproduce:
This is reproducible roughly 65% of the time.

Steps:

Requires a wired connection and a wireless card.

1. Connect your computer to your wired internet. Start your app.
2. Pull the plug and start walking away from your router.
3. Try some action that requires a realtime database transaction.
4. Observe Error: maxretry

I know that this is an incredibly contrived example, but I also believe this can happen on an unstable 4g hotspot connection. The steps above are the only way I could reliably get it to happen. Here are some relevant logs:

```
[2022-08-10 14:12:10.573] [info] @firebase/database: c:0:0:0 WebSocket error. Closing connection.
[2022-08-10 14:12:10.573] [info] @firebase/database: c:0:0:0 Network error: wss:[REDACTED]: read ETIMEDOUT
[2022-08-10 14:12:10.574] [info] @firebase/database: c:0:0:0 WebSocket is closing itself
[2022-08-10 14:12:10.574] [info] @firebase/database: c:0:0: Realtime connection lost.
[2022-08-10 14:12:10.575] [info] @firebase/database: c:0:0: Closing realtime connection.
[2022-08-10 14:12:10.575] [info] @firebase/database: c:0:0: Shutting down all connections
[2022-08-10 14:12:10.575] [info] @firebase/database: p:0: data client disconnected
[2022-08-10 14:12:10.581] [info] @firebase/database: 0: onDisconnectEvents
[2022-08-10 14:12:10.582] [info] @firebase/database: c:0:0:0 Websocket connection was disconnected.
[2022-08-10 14:12:10.583] [info] @firebase/database: p:0: Making a connection attempt
[2022-08-10 14:12:10.584] [info] @firebase/database: getToken() completed. Creating connection.
[2022-08-10 14:12:10.585] [info] @firebase/database: c:0:1: Connection created
[2022-08-10 14:12:10.586] [info] @firebase/database: c:0:1:0 Websocket connecting to wss:[REDACTED]
[2022-08-10 14:12:10.746] [info] @firebase/database: c:0:1:0 Websocket connected.
[2022-08-10 14:12:10.749] [info] @firebase/database: c:0:1: Realtime connection established.
[2022-08-10 14:12:10.749] [info] @firebase/database: p:0: connection ready
[2022-08-10 14:12:10.752] [info] @firebase/database: event: /.info/connected:value:true
[2022-08-10 14:12:10.799] [info] @firebase/database: c:0:1: Primary connection is healthy.
[2022-08-10 14:12:10.800] [info] @firebase/database: p:0: from server: {"r":23,"b":{"s":"ok","d":{}}}
[2022-08-10 14:12:10.800] [info] @firebase/database: p:0: listen response {"s":"ok","d":{}}
[2022-08-10 14:12:10.801] [info] @firebase/database: p:0: from server: {"r":24,"b":{"s":"ok","d":""}}
[2022-08-10 14:12:10.801] [info] @firebase/database: p:0: p response {"s":"ok","d":""}
[2022-08-10 14:12:10.801] [info] @firebase/database: p:0: from server: {"r":25,"b":{"s":"datastale","d":"Transaction ETag does not match"}}
[2022-08-10 14:12:10.801] [info] @firebase/database: p:0: p response {"s":"datastale","d":"Transaction ETag does not match"}
[2022-08-10 14:12:10.801] [info] @firebase/database: 0: transaction put response {"path":"[REDACTED]","status":"datastale"}
[2022-08-10 14:12:10.802] [info] @firebase/database: p:0: from server: {"r":26,"b":{"s":"ok","d":""}}
[2022-08-10 14:12:10.802] [info] @firebase/database: p:0: p response {"s":"ok","d":""}
[2022-08-10 14:12:10.802] [info] @firebase/database: p:0: from server: {"r":27,"b":{"s":"ok","d":""}}
[2022-08-10 14:12:10.802] [info] @firebase/database: p:0: p response {"s":"ok","d":""}
[2022-08-10 14:12:20.965] [warn] @firebase/firestore: Firestore (9.9.0): Connection GRPC stream error. Code: 14 Message: 14 UNAVAILABLE: read ETIMEDOUT

---This gap is from our app; it takes a while to restart a connection; part of that connection is running a transaction---

[2022-08-10 14:12:34.675] [info] @firebase/database: p:0: from server: {"r":28,"b":{"s":"datastale","d":"Transaction ETag does not match"}}
[2022-08-10 14:12:34.676] [info] @firebase/database: p:0: p response {"s":"datastale","d":"Transaction ETag does not match"}
[2022-08-10 14:12:34.677] [info] @firebase/database: p:0: {"r":29,"a":"p","b":{"p":"[REDACTED]","d":{"start":1660155130587,"stop":{".sv":"timestamp"}},"h":"[REDACTED]"}}
[2022-08-10 14:12:34.716] [info] @firebase/database: p:0: from server: {"r":29,"b":{"s":"datastale","d":"Transaction ETag does not match"}}
[2022-08-10 14:12:34.716] [info] @firebase/database: p:0: p response {"s":"datastale","d":"Transaction ETag does not match"}
[2022-08-10 14:12:34.716] [info] @firebase/database: 0: transaction put response {"path":[REDACTED],"status":"datastale"}
```

The put request + datastale runs 25 times (maxretry), then the request errors out. It keeps doing this until you restart our application, which creates a new Database object.

Contributor guide

Open the contributing guide

Research direction

The issue names no source files, tests, or entry points. Start by reproducing the wired-to-wireless transition against Realtime Database transactions and trace the reconnect handling described in the logs; done means the transaction does not remain in a datastale/maxretry loop after the connection recovers.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.