firebase / firebase/firebase-admin-python

Database Reference.listen() is very slow for large responses

Đang mở
#599 0 bình luận 0 reaction 0 người được giao Xem trên GitHub
api: database
Ngôn ngữ chính
Python
Star
1.2k
Fork
359
Merge trung bình
5 ngày 6 phút
Pull request đã merge (30 ngày)
2

Mô tả

### Describe your environment

* Operating System version: Linux raspberrypi 4.19.97+ (On a Raspberry Pi Zero W)
* Firebase SDK version: 5.2.0
* Firebase Product: database
* Python version: 3.7.3
* Pip version: 18.1

### Describe the problem

Reading a large event in response to calling Reference.listen() is extremely slow on a Raspberry Pi Zero. This looks very similar to https://github.com/nhorvath/Pyrebase4/issues/45 and I noticed that the sseclient here is based on that from Pyrebase.

From reading the sseclient code, the Firebase Admin SDK seems to have solved the quadratic regex scanning issue that the sseclient in Pyrebase4 has, but it still reads the response one character at a time, which I suspect is the root issue. It looks like this is solved in https://github.com/btubbs/sseclient/blob/master/sseclient.py by reading in chunks when short reads are supported.

#### Steps to reproduce:

Modify the test code below to add a working databaseURL and run it. On my Raspberry Pi Zero W, it outputs:

```
0 ms: Initializing app...
1 ms: Getting DB reference...
17 ms: Putting 20000 bytes...
1122 ms: Getting data...
1347 ms: Got 20000 bytes with get().
1349 ms: Listening for data...
11686 ms: Got 20000 bytes with listen().
```

That is, listen() takes about 10 000 ms (with close to 100% CPU usage) to get and read the response, while get() takes 200 ms.

#### Relevant Code:

```
import firebase_admin
import firebase_admin.db
import sys
import time

count = 2000
nonsense = "abcedefghi"
start_time = time.time()

def log(msg):
t = int(1000 * (time.time() - start_time))
print(str(t) + " ms: " + msg)

log("Initializing app...")
firebase_admin.initialize_app(options={'databaseURL': 'https://XXXXXXXXXX.firebaseio.com/'})

log("Getting DB reference...")
ref = firebase_admin.db.reference("bringert_test_streaming_speed")

data = nonsense * count

log("Putting " + str(len(data)) + " bytes...")
new_data=ref.push(data)

log("Getting data...")
log("Got " + str(len(new_data.get())) + " bytes with get().")

def callback(event):
if event.event_type == "put":
log("Got " + str(len(event.data)) + " bytes with listen().")
sys.exit(0)

log("Listening for data...")
new_data.listen(callback)
```

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.