llvm / llvm/llvm-project

[MLIR] `-split-input-file` support for `mlir-runner`

Open
#170,255 1 comment 0 reactions 0 assignees View on GitHub
mlir
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

### Summary

`mlir-opt` supports the `-split-input-file` option, which makes it convenient to keep multiple similar tests/benchmarks in a single `.mlir` file. However, `mlir-runner` does not support processing such split-input-file style inputs, which makes it hard to write integration tests that mirror how we usually structure MLIR tests.

In particular, when writing integration tests that run something like:

```sh
mlir-opt pipeline.mlir | mlir-runner ...
```

there is currently no “normal” way to handle multiple `// -----`-separated modules in a single file. We can hack around it with POSIX tools, but it would be much nicer if `mlir-runner` could consume split inputs directly.

### Current behavior

* For unit tests, it’s common to have many similar tests in one file, e.g.:

```sh
mlir-opt %s -split-input-file -some-pass ...
```

`mlir-opt` automatically splits on `// -----` markers and runs the pass pipeline per chunk.

* For integration tests, we typically do something like:

```sh
mlir-opt %s -some-pass... | \
mlir-runner -e main -entry-point-result=void ...
```

* If `input.mlir` contains multiple `// -----`-separated modules (for benchmarking multiple similar cases in one file), there is no built-in way to have `mlir-runner` execute each split chunk. `mlir-runner` expects a single module on stdin.

* The only way today is to:

* either manually split the file,
* or use shell tricks / POSIX tools to emulate `-split-input-file`, and then call `mlir-runner` per split file.

This works but is awkward and makes integration tests less straightforward and less portable.

### Expected / desired behavior

It would be very helpful if `mlir-runner` could consume split-input-file style inputs directly, similar to `mlir-opt`. For example, one of the following would work:

1. **`-split-input-file` support in `mlir-runner` itself**, so we could write:

```sh
mlir-opt -pass-pipeline='...' my_benchmarks.mlir \
| mlir-runner -split-input-file -e main -entry-point-result=void ...
```

and have `mlir-runner` execute each chunk independently.

2. Or, some other option / mechanism that allows `mlir-runner` to:

* accept multiple modules separated by `// -----` on stdin, and
* either run them sequentially, or fail with a clear error if that’s not supported.

The key point is: **I want to keep multiple similar benchmarks in a single `.mlir` file and still be able to run them via `mlir-runner` in a “standard” way, without resorting to custom splitting scripts.**

Contributor guide

Open the contributing guide

Research direction

Start by examining the existing -split-input-file handling in mlir-opt and the stdin/module processing path in mlir-runner. Determine how // ----- chunks are represented and define the integration-test behavior for running multiple modules sequentially or reporting an unsupported input clearly.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.