jackson211 / jackson211/rust_logger_benchmark
Some improvement ideas
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 13
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Some ideas based on looking at the code and results
Create set of library functions in src so that main.rs and the benchmarks use the same configuration for each logger. Right now the config is duplicated and inconsistent, so what you see in main when looking at each logger is inconsistent.
Create a single main criterion benchmark instead of multiple benches. This allows the criterion report to contain all the information and makes it easier to compare the violin plots between the various frameworks. A blocker to this is that setting which logger is in use is (mostly) a one time thing. Some frameworks do allow for a guard style which makes it possible to reset logging when dropped. You may be able to get around this in 2 ways:
- Make each logger into a small cli and call that from the benches (this likely has some weird problems, but might be possible to mitigate)
- Configure each logger that can use the guard approach to do so, configure the other loggers with some sort of shim which dispatches to the configured logger at runtime (this likely has an overhead, but could be baselined against a dummy dropping logger)
Take the terminal out of the equation - logging to stdout means that the specific terminal used (or not used) will have a meaningful effect on the benchmark. Configure benchmarks to write to a discarding Sink. Also configure them to write to an in memory buffer to be able to compare the size / count of messages. It's likely that measuring bytes per second instead of just message count will highlight that much of the differences in speed can be explained by the size of the output. (This has a side benefit of making the criterion results easier to read)
Handle async / dropping correctly. Providing results that don't highlight that the slog and ftlog results are dropping a significant amount of log messages is misleading.
Document the configuration goals. There's probably a few competing criteria:
- What's the performance of the default / idiomatic configuration of the logger (i.e. what's in the box)
- What's the performance when the logger is configured to report the same or similar information. Find a common format that helps avoid comparing apples and oranges. The following are the obvious items which impact the timings quite a bit:
- timestamp precision / formatting
- timezone: local (static or detected) / utc (should be the default generally)
- ANSI formatting: mostly affects levels, but tracing has colors throughout its default output
- dropping messages on overload
- What information about the target / name / location is logged
Add more specific comparisons for:
- key value support
- spans / target / file info
- timezone
- timestamp source
- ansi state
Add tests for logging to a file. This also allows stats about log size to be compared.
Parameterize the benchmark iteration counts so that the time per benchmark can be reduced. While the default values are good for being statistically comprehensive, they're terrible for iterating on benchmarks to make them consistent / fast as the cycle time takes a hit.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading src, main.rs, and the benchmark files to map the duplicated logger configuration and current Criterion setup. Break the ideas into separately reviewable changes, beginning with shared configuration and controlled benchmark sinks. Done requires consistent logger comparisons, correct handling of dropped messages, documented configuration goals, and tests or measurements for the requested cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- performance, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100