dart-lang / dart-lang/native

[jnigen] Benchmarks

Open
#718 3 comments 3 reactions 0 assignees View on GitHub
package:jni package:jnigen
Dominant language
Dart
Stars
275
Forks
144
Avg merge
2d 10h
Merged PRs (30d)
47

Description

We should add benchmarks to this repo.

- `jni/benchmark` and `jnigen/benchmark`
- Use https://pub.dev/packages/benchmark_harness
- You can see some examples https://github.com/dart-lang/sdk/tree/main/benchmarks/FfiMemory
- You can see some examples https://github.com/dart-lang/sdk/tree/main/benchmarks/FfiCall
- (But please don't copy the non-Dart-style Dart!)
- They should be run on CI (just like the `jni(gen)/example`) to ensure they work (at least the Dart standalone ones)
- We should explore if there is a way to do benchmarks on CI.
- https://github.com/benchmark-action has no Dart support yet, but if the interface is json or something we might still be able to use it.

We should include benchmarks with method channels to see how the performance compares:

* jnigen with generating C code
* jnigen with generating only Dart code
* method channels

Unfortunately, running a Flutter app is hard without a window manager, so that might make it impossible to run the method channels benchmarks in an automated fashion.

So, besides having trying to run the benchmarks on CI, we should probably also upload benchmark results manually to the repo. (Extra kudos for making the benchmarks just generate json/yaml as output and a script that takes all the benchmarks, collects all the outputs and generates a benchmark_results.md so that updating the numbers is trivial later on.)

Fur running flutter apps from CLI `flutter drive`. (Or use `flutter run` capture stdout and just `exit(0);` when done with benchmarking.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.