google / google/heir

Get slowest test to run in < 1 minute

Open
#2,088 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
MLIR
Stars
906
Forks
171
Avg merge
4d 12h
Merged PRs (30d)
32

Description

To get a list of the slowest tests in HEIR, run

```bash
bazel test //... --nocache_test_results --test_summary=detailed 2>&1 > test_summary.txt
grep '^//' test_summary.txt | awk '{ print $1, $4}' | sort -k2 -nr | head -n 25
```

We should ensure that the slowest test run by default `bazel test //...` runs in under 1 minute. I think this is a decent target to start, at least because the frontend tests are limited by the speed of the linker which we know is a bottleneck.

After https://github.com/google/heir/pull/2087, the slowest tests (as run on my beefy desktop) are

```
//tests/Examples/tfhe_rust:sbox_test 248.2s
//tests/Examples/tfhe_rust_bool/cpu:bool_add_test 170.2s
//tests/Examples/tfhe_rust_hl/cpu:hello_world_clean_xsmall_test 160.7s
//tests/Examples/tfhe_rust_hl/cpu:arith_test 149.9s
//tests/Examples/tfhe_rust:add_one_test 140.6s
//tests/Transforms/mlir_to_tfhe_rs:hello_world_small.mlir.test 139.0s
//frontend:e2e_test 131.1s
//tests/Examples/tfhe_rust_hl/cpu:add_round_key_test 124.6s
//tests/Examples/tfhe_rust:multi_output_test 110.4s
//tests/Transforms/yosys_optimizer:micro_speech_for.mlir.test 94.8s
//tests/Examples/openfhe/ckks/halevi_shoup_matvec:halevi_shoup_matvec_test 60.8s
//tests/Examples/tfhe_rust_hl/cpu:fully_connected_test 59.7s
//frontend:tensor_test 52.0s
//frontend:loop_test 51.3s
//frontend:mixed_bitwidth_test 45.1s
//frontend:tensor_loop_test 41.5s
//tests/Transforms/mlir_to_tfhe_rs:variance.mlir.test 34.0s
//tests/Regression:issue_1929.mlir.test 28.5s
//tests/Examples/tfhe_rust:simple_lut_test 26.3s
//tests/Examples/openfhe/ckks/simple_ckks_bootstrapping:simple_ckks_bootstrapping_test 20.7s
//tests/Examples/tfhe_rust:bitand_test 19.3s
//tests/Transforms/heir_simd_vectorizer:box_blur_16x16.mlir.test 19.2s
//tests/Transforms/heir_simd_vectorizer:gx_kernel_16x16.mlir.test 17.1s
//frontend:cggi_test 16.8s
//tests/Transforms/yosys_optimizer:many_inputs.mlir.test 15.4s
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.