google / google/xls

IR benchmark failure

Open
#1,733 3 comments 0 reactions 0 assignees View on GitHub
🐜 ant ir
Dominant language
C++
Stars
1.9k
Forks
283
Avg merge
2d 10h
Merged PRs (30d)
135

Description

**Describe the bug**
The IR benchmark for [DecoderMux](https://github.com/google/xls/blob/cd901120adfa5355b5ea40adcd97bfc61af2f3c7/xls/modules/zstd/dec_mux.x) proc is prone to a random failure after increasing [pipeline_stages](https://github.com/google/xls/blob/cd901120adfa5355b5ea40adcd97bfc61af2f3c7/xls/modules/zstd/BUILD#L523) parameter for the benchmark from `2` up to `10`. The error is indicated by the following output:
```
/home/xlsuser/.cache/bazel/_bazel_xlsuser/976f9331675cc2c7ada78a437e0227fe/execroot/com_google_xls/bazel-out/k8-opt/bin/xls/modules/zstd/dec_mux_opt_ir_benchmark.sh: line 3: 3086416 Segmentation fault (core dumped) xls/dev_tools/benchmark_main xls/modules/zstd/dec_mux_verilog.opt.ir --delay_model=asap7 --generator=pipeline --pipeline_stages=10 --reset=rst --reset_data_path=false --use_system_verilog=false --module_name=dec_mux_opt_ir_benchmark_default $@ $*
```

This failure does not occur for every benchmark run. It might be required to run the benchmark multiple times (could even take 50 runs) to reproduce the error.

**To Reproduce**
Steps to reproduce the behavior:
1. Checkout on current `main` branch (https://github.com/google/xls/commit/cd901120adfa5355b5ea40adcd97bfc61af2f3c7 at the time of writing this issue)
2. Modify `pipeline_stages` parameter for `DecoderMux` codegen and IR benchmark:
```
diff --git a/xls/modules/zstd/BUILD b/xls/modules/zstd/BUILD
index 871749792..94b6b3ecd 100644
--- a/xls/modules/zstd/BUILD
+++ b/xls/modules/zstd/BUILD
@@ -506,7 +506,7 @@ xls_dslx_verilog(
codegen_args = {
"module_name": "DecoderMux",
"delay_model": "asap7",
- "pipeline_stages": "2",
+ "pipeline_stages": "3",
"reset": "rst",
"use_system_verilog": "false",
},
@@ -520,7 +520,7 @@ xls_benchmark_ir(
name = "dec_mux_opt_ir_benchmark",
src = ":dec_mux_verilog.opt.ir",
benchmark_ir_args = {
- "pipeline_stages": "2",
+ "pipeline_stages": "10",
"delay_model": "asap7",
},
tags = ["manual"],
```
2. Run `bazel run -c opt -- //xls/modules/zstd:dec_mux_opt_ir_benchmark` repeatedly up to the segfault occurrence, e.g.:
```
count=0; while bazel run -c opt -s -- //xls/modules/zstd:dec_mux_opt_ir_benchmark --logtostderr; do (( count++ )); echo "Run $count"; done; echo "successfull runs: $count"
```
3. Observe segfault of the `benchmark_main`

**Expected behavior**
`benchmark_main` should fail gracefully or not at all

**Additional context**
I noticed that proc `DecoderMux` can be divided by the toolchain into more than 2 pipeline stages as it is currently on the `main` branch. I increased this parameter for the IR benchmark to 10 to see the number of stages that have some IR nodes assigned to those (happens to be 3 pipeline stages). Then I set the pipeline stages for the codegen rule to 3 stages indicated by the benchmark. Then I noticed that sometimes the IR benchmark would fail.

The segfault seems to be happening somewhere in [the evaluation of the Block IR with JIT](https://github.com/google/xls/blob/cd901120adfa5355b5ea40adcd97bfc61af2f3c7/xls/dev_tools/benchmark_main.cc#L656).

This was also happening in the CI for the https://github.com/google/xls/pull/1616: https://github.com/google/xls/actions/runs/11494086318/job/32320996820?pr=1616#step:10:463

CC @proppy

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.