apache / apache/solr-orbit

[Bug]: converted workloads abort on "operation-type": "index-stats", which has no runner

Open
#79 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3
Forks
5
Avg merge
1d 23m
Merged PRs (30d)
5

Description

### Description

A converted workload aborts on the task that waits for merges to finish. OpenSearch Benchmark has no operation type for that wait — workloads express it as an `index-stats` call retried until the active merge count reaches zero:

```json
{
"name": "wait-until-merges-finish",
"operation-type": "index-stats",
"index": "_all",
"condition": { "path": "_all.total.merges.current", "expected-value": 0 },
"retry-until-success": true,
"include-in-reporting": false
}
```

The converter copies `"operation-type": "index-stats"` through unchanged, and no runner is registered for it, so `runner_for` raises and the run stops.

Solr Orbit already has the operation this is meant to be: `wait-for-merges`, registered since the initial port (`57387ed4`), whose runner polls node metrics in its own loop. The hand-written Solr `nyc_taxis` in `solr-orbit-workloads` uses it. So this is not a missing feature — the converter simply does not reach for it.

### To reproduce

On `main` at f0883807, convert upstream `nyc_taxis` and run that task on its own against the default test procedure:

```
solr-orbit run --workload-path /nyc_taxis --pipeline=benchmark-only \
--target-hosts=localhost:8983 --include-tasks=wait-until-merges-finish
```

```
[ERROR] ❌ Cannot run. Benchmark operation failed:
Worker ID: 0
Task: wait-until-merges-finish
Workload: nyc_taxis
Test Procedure: append-no-conflicts
Cause: Unknown
Error: No runner available for operation type [index-stats]
No runner available for operation type [index-stats]
```

Exit code 64.

### Expected behavior

An `index-stats` operation whose condition polls the merge count to zero should be converted to `wait-for-merges`, with the polling OpenSearch expresses through `condition` and `retry-until-success` carried by the runner's own `retry-wait-period` and `max-wait-seconds`.

The hand-written Solr `nyc_taxis` in `solr-orbit-workloads` already states the intended result, so there is a target to compare against:

```json
{
"name": "wait-until-merges-finish",
"operation-type": "wait-for-merges",
"retry-wait-period": 2.0,
"max-wait-seconds": 600,
"include-in-reporting": false
}
```

### Additional context

In the default test procedure the merge wait arrives through `common_operations/force_merge.json`, which is collected after `index` and `refresh-after-index` — so a full converted `nyc_taxis` run fails only once the corpus is indexed and merged. The expensive part is already spent when it aborts.

It is not specific to `nyc_taxis`. Because the idiom lives in that shared fragment, it reaches the converted output of `clickbench`, `geoshape`, `http_logs`, `neural_search`, `nyc_taxis`, `pmc` and `treccovid_semantic_search` — 16 operation definitions across 13 generated files, all of them this idiom.

A 17th match in `geonames/test_procedures/default.json` is a different thing and not part of this report: `"operation": "index-stats"` there is a schedule reference to OpenSearch Benchmark's built-in operation of that name, which the workload never defines. It also has no Solr runner, but the fix is not the same one.

Two further source occurrences are out of reach for a third reason — `clickbench/test_procedures/common/snapshot-ops.json` and `noaa_semantic_search/test_procedures/semantic-search-common/check-status-add-resources-no-index.json` sit in nested directories under `test_procedures/`, and the converter only walks the top-level `.json` files there, so those files are absent from the output entirely. That is a separate defect; I will report it on its own.

### Version

`main` at f0883807.

Contributor guide

Open the contributing guide

Research direction

Trace the converter's handling of operation-type values and compare its generated output with the hand-written Solr nyc_taxis workload in solr-orbit-workloads. Check the shared common_operations/force_merge.json input and the 16 affected definitions; done means merge waits become wait-for-merges and the supplied nyc_taxis command completes without the missing-runner error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.