Extend proc evaluator
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 283
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 135
Description
Currently we have a tool for evaluating proc (eval_proc) but it's pretty limited in scope at the moment. For example, you can't specify inputs to feed to the proc. We also want to support the functionality available in eval_ir_main like evaluating the proc after each optimization pass, comparing JIT and interpreter results, etc. So I think it may make sense to fold eval_proc functionality into eval_ir_main and extend the functionality.
Currently eval_ir_main assumes it's evaluating a function with potentially multiple inputs and one output. This means inputs/outputs are specified via flags like so:
```
--input='bits[32]:42; (bits[7]:0, bits[20]:4)' # Values to pass to two-argument function
--expected='bits[32]:42' # expected output
--random_inputs=100 # randomly generate 100 sets of arguments
```
We would probably need to generalize these to allow specifying inputs and expected outputs for channels. Maybe:
```
--input='a:{bits[32]:42; bits[32]:0}, b:{bits[7]:0}' # Values to pass to input channels "a" and "b"
--expected='out0:{bits[32]:42}, out1:{bits[1]:0, bits[1]:1}' # expected values from output channels
--random_inputs='a:100, b:3' # randomly generate 100 inputs for channel "a" and 3 for channel "b"
```
Procs are interesting because there is no indication that the proc is done processing it's inputs and generating outputs. We would probably handle this for ~all realistic cases by stopping the proc when everything is just waiting on inputs (and all inputs have already been fed), or a large number of ticks have passed (e.g, 100K or something). Timing out due to too many ticks could be an error.
A proc may generate more than the expected number of outputs. We probably want to allow this, for example, to allow matching the first N outputs of a proc. Though we also probably want to flag an error in some cases too where we only want exactly N outputs for the given inputs. this could be controlled by a flag.
Also thinking forward to evaluating to blocks which allow multiple outputs. We could do something like:
```
--input='a:bits[32]:42, b:(bits[7]:0, bits[20]:4)' # Values to pass to ports "a" and "b"
--expected='out1:bits[32]:42, out2:bits[1]:0' # expected outputs from ports "out1" and "out2"
--random_inputs=100 # Behaves just like for functions
```
Contributor guide
Assessment
This issue has not been assessed yet.