HarperFast / HarperFast/harper
Untyped REST numeric conditions on GraphQL-declared array attributes silently return empty — coerceType has no 'array' case
- Dominant language
- JavaScript
- Stars
- 89
- Forks
- 10
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 200
Description
## Summary
An **untyped** REST numeric comparison against an attribute declared as an array in a GraphQL schema (`sizes: [Int]`) silently returns an **empty result set** — HTTP 200, no error, no warning. The identical condition against a scalar `Int` attribute holding the same numbers returns the correct rows, and the `number:`-prefixed form of the same query against the same array attribute also returns the correct rows.
This reproduces on **both** the `@indexed` and the unindexed array attribute, so it is a *coercion* defect, not an index defect — distinct from #2434 (duplicate rows from indexed `elements` attributes), which is about multiplicity on a query that does match.
## Reproduction
harper `93752ad79` (main, v5.3 line; originally measured on `2e65550d7`, mechanism unchanged), default RocksDB engine, no non-default config.
```
npm run test:integration -- "integrationTests/qa-scratch/qa922-elements-coercion.test.ts"
```
Schema: `sizes: [Int] @indexed`, `sizesPlain: [Int]`, `scalarIdx: Int @indexed`, `scalarPlain: Int`. 25 records; every record's scalar attributes are >= 300 and every record has at least one array element >= 300, so `ge 300` is correct for all 25 records on all four attributes.
| query | observed | expected |
|---|---|---|
| `?scalarIdx=ge=300` | 25 ids | 25 ids |
| `?scalarPlain=ge=300` | 25 ids | 25 ids |
| `?sizes=ge=300` | **[]** | 25 ids |
| `?sizesPlain=ge=300` | **[]** | 25 ids |
| `?=ge=number:300` | 25 ids | 25 ids |
Verified over 3 cold runs (1 builder + 2 independent fresh-context verifier runs), identical every time.
## Mechanism
- `resources/Table.ts:3511-3512` (`prepareConditions`) is the only site that coerces a condition value, via `coerceTypedValues` (`resources/Table.ts:3561-3565`) → `coerceType(value, attribute)`.
- `resources/Table.ts:7007` `coerceType()` switches on `attribute.type`. There is **no `'array'` case**, so it falls through to `default: return value` at `resources/Table.ts:7046-7047` and the FIQL string is never converted to a number.
- Execution compares via `compareKeys` (`ordered-binary/index.js:395-436`), whose cross-type fallback is `typeOrder[typeof a] < typeOrder[typeof b]` with `number: 3 < string: 4` — **every number sorts below every string regardless of magnitude** — so a string lower bound excludes every record.
### Why this escaped: the declaration route decides
`resources/graphql.ts:158` maps a GraphQL `ListType` to `{ type: 'array', elements: getProperty(...) }`, so a `.graphql`-declared array attribute has `attribute.type === 'array'` → **broken**.
A **programmatic** declaration `{ name: 'sizes', elements: { type: 'Int' } }` omits the top-level `type` key entirely, so `attribute.type === undefined` → `coerceType`'s `case undefined: return autoCast(value)` (`resources/Table.ts:7043-7045`) coerces `"300"` → `300` → **works**.
This is why #2433's verification PR #2437 does not show it: that fixture uses the programmatic form. The documented, user-facing schema route is the broken one; the internal test route is the working one.
## Candidate fix (validated by fault injection)
```js
case 'array':
return coerceType(value, attribute.elements);
```
Injecting this into the build artifact made all four attributes return the correct 25 records under the untyped query. (It also flipped the attached regression spec red, as intended — see "Tests" below.)
## Existing work checked
Searched `HarperFast/harper` and `HarperFast/harper-pro`, issues **and** PRs, open **and** closed, over: `coerc*`, `elements`, `array query`, `untyped`, `fiql`, `rest condition`, `silent empty`, `empty result`, `type prefix`, `number:`.
- **#2433** (OPEN) — *closest sibling, and this is NOT a duplicate of it.* #2433 is the **chained-leg** gap: `prepareConditions` coerces `condition.value` but never `condition.chainedConditions[i].value`. This finding is the **single, unchained** leg failing on an array attribute because `coerceType` has no `'array'` case. Same subsystem, same silent-wrong-results signature, different code defect; fixing one does not fix the other. #2433's analysis explicitly presupposes the first leg *was* coerced — true for its programmatic fixture, false for a GraphQL-declared attribute.
- **#2434** (OPEN) — indexed `elements` duplicate rows. Different defect; that query matches and over-returns, this one silently matches nothing, and this reproduces on the unindexed attribute too.
- **#2200**, **#1897** — coercion in Joi validation and component options. Unrelated surfaces.
- **#1482** (OPEN, `?select(singleField)` silently returns empty) — same *class* of silent-empty REST defect, different cause.
- Nothing in `harper-pro`. Nothing tracks this.
## What this does NOT prove
- **RocksDB only** (default engine); no `HARPER_STORAGE_ENGINE=lmdb` arm.
- **Single-node**; no cluster or replication.
- **Comparator coverage is source-derived, not measured.** Predicted but not re-run: `equals` (`resources/search.ts:956`, strict `===`) should fail the same way; `ct`/contains (`:959`, `toString().includes`) should be unaffected; a `[String]` array should be unaffected, since leaving the value a string is correct there. Queued as a follow-up scenario.
- **Only two declaration routes checked** — GraphQL `[Int]` (→ `'array'`) and a literal `elements`-only object (→ `undefined`). Whether JSON Schema, MCP-generated schemas, or programmatic `table({...})` in product code produce one or the other is unverified, and that is what sizes the blast radius.
- **Not bisected.** "Long-shipped" is inference from the code shape, not a `git bisect`.
## Tests
A regression spec exists at `state/promote-candidates/P-651/qa922-elements-coercion.test.ts` (qa-explorer). It currently asserts the **defect** and will go red when this is fixed — it should be inverted as part of the fix rather than promoted as-is.
## Re-verified 2026-09-02 on `93752ad79`
`coerceType()` still has no `case 'array'` and still falls through to `default: return value`; `resources/graphql.ts` still maps a GraphQL `ListType` to `{ type: 'array', elements: ... }`. Unchanged.
PR #2437 ("Pin element-scoping semantics of queries over array-valued properties", merged 2026-09-01) added `resources/DESIGN.md` and `unitTests/resources/query-array-scoping.test.js` plus the tracking epic #2440 — **documentation and regression scaffolding only, no production code**. Its fixture declares `{ name: 'sizes', elements: { type: 'Int' } }` with no top-level `type` key, which is exactly the programmatic route that hits `case undefined: return autoCast(value)` and therefore works. That is why the new pins do not show this defect.
#2440's divergence table (rows 1-14, "as of harper main, 2026-09") lists #2433 at row 10 and #2434 at row 11. **This mechanism — a single, unchained condition on a GraphQL-declared array attribute — is not a row in that table**, so it is currently tracked nowhere.
Contributor guide
Research direction
Start by running integrationTests/qa-scratch/qa922-elements-coercion.test.ts and inspect resources/Table.ts around prepareConditions and coerceType, then compare the GraphQL array declaration in resources/graphql.ts. Update the regression spec at state/promote-candidates/P-651/qa922-elements-coercion.test.ts so it asserts the expected 25 records, and confirm the untyped numeric array queries pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, javascript
- Domain
- api, database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100