Generation is very slow in `replay-file` mode
- Dominant language
- Go
- Stars
- 14
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the bug**
We are trying to generate logs using a predefined pattern and use `replay-file` for this. But looking into the output, we see that generation rate is very slow ~ 600 line per second.
**To Reproduce**
in.log
```
209.124.125.232 - franecki2473 [22/Dec/2022:18:56:54 +0000] "GET /value-added/repurpose/partnerships/cultivate HTTP/2.0" 403 5187
```
```bash
loadgen replay-file \
--duration 5 \
--input-file-path in.log \
--output out.log \
--result-log result.log \
--lines-per-second 20000 \
--enable-metrics
```
**Expected behavior**
Application should be able to generate logs with a higher speed
**Desktop:**
- OS: m5a.large / Ubuntu 22.04
**Additional context**
```bash
while true; do
processed=$(curl http://localhost:8080/metrics -s | awk '/^total_events_processed/ {print $2}')
[[ -z ${precedent} ]] && precedent=0
rate=$((processed-precedent))
echo "`date` - ${processed} - ${rate}"
sleep 1
precedent=${processed}
done
```
```
Thu Dec 22 20:33:18 UTC 2022 - 353 - 353
Thu Dec 22 20:33:19 UTC 2022 - 1639 - 1286
Thu Dec 22 20:33:20 UTC 2022 - 2924 - 1285
Thu Dec 22 20:33:21 UTC 2022 - 4221 - 1297
Thu Dec 22 20:33:22 UTC 2022 - 5497 - 1276
Thu Dec 22 20:33:23 UTC 2022 - 6799 - 1302
Thu Dec 22 20:33:24 UTC 2022 - 8083 - 1284
Thu Dec 22 20:33:25 UTC 2022 - 9385 - 1302
Thu Dec 22 20:33:26 UTC 2022 - 10671 - 1286
Thu Dec 22 20:33:27 UTC 2022 - 11955 - 1284
Thu Dec 22 20:33:28 UTC 2022 - 13251 - 1296
Thu Dec 22 20:33:29 UTC 2022 - 14551 - 1300
Thu Dec 22 20:33:30 UTC 2022 - 15847 - 1296
Thu Dec 22 20:33:31 UTC 2022 - 17133 - 1286
Thu Dec 22 20:33:32 UTC 2022 - 18427 - 1294
```
In the metrics we see that rate is ~ 1200 line per seconds, but if we look into the file
```bash
cat out.log | grep 2022-12-22T20:33:33 | wc -l
634
```
we can see just half of the number compared to the metric one.
Contributor guide
Research direction
Start with the replay-file mode using the exact loadgen command and input shown, then compare total_events_processed with the lines written to out.log. Investigate why the observed generation rate is far below --lines-per-second and why metrics exceed the matching output lines; done means substantially faster generation with consistent metrics and output counts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100