apache / apache/solr-orbit

[Bug]: query randomization has no effect on Solr workloads — the query is a string, not an object tree

Open
#75 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3
Forks
5
Avg merge
1d 23m
Merged PRs (30d)
5

Description

### Description

`--randomization-enabled` has no effect on a Solr workload. Every client sends the
one query written in the workload's `operations/*.json`, for the whole run, and the
log says randomization is on.

`QueryRandomizerWorkloadProcessor.extract_fields_and_paths` walks `params["body"]["query"]`
as an object tree looking for a `range` key, and `set_range` writes the new bounds back
into that object. That is the OpenSearch query DSL, where the query is a nested object.
Solr's JSON DSL states the query as a *string*:

```json
{ "query": "total_amount:[5 TO 15}" }
```

so the walk reaches a leaf immediately and returns no fields. With no fields, no standard
value source is called, `set_range` never runs, and `get_randomized_values` returns the
body it was given.

### Reproduction

On `main` at f0883807, with the two shapes of the same nyc_taxis `range` operation:

```
Solr JSON DSL -> []
OpenSearch query DSL -> [('total_amount', ['bool', 'filter', 'range'])]
```

and end to end:

```
get_randomized_values -> total_amount:[5 TO 15} # the value source returned {"gte": 3, "lte": 7}
```

Meanwhile `loader.py:1205` logs

```
Query randomization is enabled, with repeat frequency = 0, n = 5000
```

### Effect

Every nyc_taxis operation that registers a standard value source is affected — `range`,
`distance_amount_facet`, `date_histogram_facet`, `date_histogram_calendar_interval`,
`date_histogram_fixed_interval`. A run made with `--randomization-enabled` measures the
same five queries as a run made without it, so it reports a warmed, single-query cache
where it is meant to report a spread. Nothing fails, and no output distinguishes the two
runs.

It also means every defect in a workload's value sources is unreachable, and so untested:
a value source is only ever called from this processor. In `solr-orbit-workloads`
`nyc_taxis/workload.py` there are two, both latent today — `random_money_values` passes a
float to `random.randrange`, which raises `TypeError` on every Python this project
supports, and `date_source_without_hours` writes both bounds at `T00:00:00Z` into a query
with an exclusive upper bound, so one draw in six matches no documents at all. I will open
those separately against `solr-orbit-workloads`; they become reachable the moment this is
fixed.

### Expected

A range term in a Solr query string should have its bounds substituted, keeping the
brackets the query already states, so an exclusive bound stays exclusive — the same way
the object path keeps whichever of `lt`/`lte` it found. A term that leaves a bound open
(`total_amount:[5 TO *]`) should be left alone, since the object path does not randomize a
one-sided range either.

### Version

`main` at f0883807.

Contributor guide

Open the contributing guide

Research direction

Start at QueryRandomizerWorkloadProcessor.extract_fields_and_paths and set_range, then inspect the Solr query strings in operations/*.json and the randomization call path. Use loader.py:1205 to verify the existing log, and confirm that two-sided bounds are substituted while one-sided ranges remain unchanged and bracket exclusivity is preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases, performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.