fluvio-community / fluvio-community/fluvio-client-python
Performance degradation on python producer.send with large number of records
- Dominant language
- Python
- Stars
- 19
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
slack thread: https://infinyon.slack.com/archives/CQ8UX6SBH/p1684349700779719
simple producer in python:
```python
#!/usr/bin/env python3
from datetime import datetime
from fluvio import Fluvio
import time
PARTITION = 0
if __name__ == "__main__":
# Connect to cluster
fluvio = Fluvio.connect()
producer = fluvio.topic_producer("simple")
start_time = time.time()
for x in range(10000):
producer.send_string("{}: timestamp: {}".format(x, datetime.now()))
# Flush the last entry
producer.flush()
print("completed in {} seconds", (time.time()-start_time))
```
10K records pushed in 1.2153611183166504 seconds, when if you increase the range to 100000
python3 producer100K.py
The same code takes 64.67606687545776 to complete.
Rust producer doesn't have such behaviour.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.