google / google/xls

Add proc evaluation time to benchmark main

Open
#749 0 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
C++
Stars
1.9k
Forks
283
Avg merge
2d 10h
Merged PRs (30d)
135

Description

benchmark_main (https://github.com/google/xls/blob/main/xls/tools/benchmark_main.cc) takes an IR file and optimizes it, evaluates it with the jit and interpreter, etc and dumps a bunch of metrics. Currently it does not support the evaluation of procs because procs may not execute properly with random values fed into the inputs (functions are fed random inputs in the benchmark script). We now have defined a file format which specifies a sequence of values for any number of channels:

https://github.com/google/xls/blob/18742d56a5089329a9eaacb15f186b385e1e167b/xls/tools/eval_helpers.h#L78

This could be used to feed the proc so we could measure the performance of interpreter/jit on the proc. Maybe could be tied into the build rules like so:

```
xls_benchmark_ir(
name = "foo_proc_benchmark_ir",
src = ":foo_proc.ir",
inputs="foo_inputs.txt"
)
```

Might need to sepcify the number of ticks to run it, or maybe just run it until one (all?) of the inputs queues is empty.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.