HarperFast / HarperFast/harper

Untyped REST numeric conditions on GraphQL-declared array attributes silently return empty — coerceType has no 'array' case

Open
#2,481 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

## Summary

An **untyped** REST numeric comparison against an attribute declared as an array in a GraphQL schema (`sizes: [Int]`) silently returns an **empty result set** — HTTP 200, no error, no warning. The identical condition against a scalar `Int` attribute holding the same numbers returns the correct rows, and the `number:`-prefixed form of the same query against the same array attribute also returns the correct rows.

This reproduces on **both** the `@indexed` and the unindexed array attribute, so it is a *coercion* defect, not an index defect — distinct from #2434 (duplicate rows from indexed `elements` attributes), which is about multiplicity on a query that does match.

## Reproduction

harper `93752ad79` (main, v5.3 line; originally measured on `2e65550d7`, mechanism unchanged), default RocksDB engine, no non-default config.

```
npm run test:integration -- "integrationTests/qa-scratch/qa922-elements-coercion.test.ts"
```

Schema: `sizes: [Int] @indexed`, `sizesPlain: [Int]`, `scalarIdx: Int @indexed`, `scalarPlain: Int`. 25 records; every record's scalar attributes are >= 300 and every record has at least one array element >= 300, so `ge 300` is correct for all 25 records on all four attributes.

| query | observed | expected |
|---|---|---|
| `?scalarIdx=ge=300` | 25 ids | 25 ids |
| `?scalarPlain=ge=300` | 25 ids | 25 ids |
| `?sizes=ge=300` | **[]** | 25 ids |
| `?sizesPlain=ge=300` | **[]** | 25 ids |
| `?=ge=number:300` | 25 ids | 25 ids |

Verified over 3 cold runs (1 builder + 2 independent fresh-context verifier runs), identical every time.

## Mechanism

- `resources/Table.ts:3511-3512` (`prepareConditions`) is the only site that coerces a condition value, via `coerceTypedValues` (`resources/Table.ts:3561-3565`) → `coerceType(value, attribute)`.
- `resources/Table.ts:7007` `coerceType()` switches on `attribute.type`. There is **no `'array'` case**, so it falls through to `default: return value` at `resources/Table.ts:7046-7047` and the FIQL string is never converted to a number.
- Execution compares via `compareKeys` (`ordered-binary/index.js:395-436`), whose cross-type fallback is `typeOrder[typeof a] < typeOrder[typeof b]` with `number: 3 < string: 4` — **every number sorts below every string regardless of magnitude** — so a string lower bound excludes every record.

### Why this escaped: the declaration route decides

`resources/graphql.ts:158` maps a GraphQL `ListType` to `{ type: 'array', elements: getProperty(...) }`, so a `.graphql`-declared array attribute has `attribute.type === 'array'` → **broken**.

A **programmatic** declaration `{ name: 'sizes', elements: { type: 'Int' } }` omits the top-level `type` key entirely, so `attribute.type === undefined` → `coerceType`'s `case undefined: return autoCast(value)` (`resources/Table.ts:7043-7045`) coerces `"300"` → `300` → **works**.

This is why #2433's verification PR #2437 does not show it: that fixture uses the programmatic form. The documented, user-facing schema route is the broken one; the internal test route is the working one.

## Candidate fix (validated by fault injection)

```js
case 'array':
return coerceType(value, attribute.elements);
```

Injecting this into the build artifact made all four attributes return the correct 25 records under the untyped query. (It also flipped the attached regression spec red, as intended — see "Tests" below.)

## Existing work checked

Searched `HarperFast/harper` and `HarperFast/harper-pro`, issues **and** PRs, open **and** closed, over: `coerc*`, `elements`, `array query`, `untyped`, `fiql`, `rest condition`, `silent empty`, `empty result`, `type prefix`, `number:`.

- **#2433** (OPEN) — *closest sibling, and this is NOT a duplicate of it.* #2433 is the **chained-leg** gap: `prepareConditions` coerces `condition.value` but never `condition.chainedConditions[i].value`. This finding is the **single, unchained** leg failing on an array attribute because `coerceType` has no `'array'` case. Same subsystem, same silent-wrong-results signature, different code defect; fixing one does not fix the other. #2433's analysis explicitly presupposes the first leg *was* coerced — true for its programmatic fixture, false for a GraphQL-declared attribute.
- **#2434** (OPEN) — indexed `elements` duplicate rows. Different defect; that query matches and over-returns, this one silently matches nothing, and this reproduces on the unindexed attribute too.
- **#2200**, **#1897** — coercion in Joi validation and component options. Unrelated surfaces.
- **#1482** (OPEN, `?select(singleField)` silently returns empty) — same *class* of silent-empty REST defect, different cause.
- Nothing in `harper-pro`. Nothing tracks this.

## What this does NOT prove

- **RocksDB only** (default engine); no `HARPER_STORAGE_ENGINE=lmdb` arm.
- **Single-node**; no cluster or replication.
- **Comparator coverage is source-derived, not measured.** Predicted but not re-run: `equals` (`resources/search.ts:956`, strict `===`) should fail the same way; `ct`/contains (`:959`, `toString().includes`) should be unaffected; a `[String]` array should be unaffected, since leaving the value a string is correct there. Queued as a follow-up scenario.
- **Only two declaration routes checked** — GraphQL `[Int]` (→ `'array'`) and a literal `elements`-only object (→ `undefined`). Whether JSON Schema, MCP-generated schemas, or programmatic `table({...})` in product code produce one or the other is unverified, and that is what sizes the blast radius.
- **Not bisected.** "Long-shipped" is inference from the code shape, not a `git bisect`.

## Tests

A regression spec exists at `state/promote-candidates/P-651/qa922-elements-coercion.test.ts` (qa-explorer). It currently asserts the **defect** and will go red when this is fixed — it should be inverted as part of the fix rather than promoted as-is.

## Re-verified 2026-09-02 on `93752ad79`

`coerceType()` still has no `case 'array'` and still falls through to `default: return value`; `resources/graphql.ts` still maps a GraphQL `ListType` to `{ type: 'array', elements: ... }`. Unchanged.

PR #2437 ("Pin element-scoping semantics of queries over array-valued properties", merged 2026-09-01) added `resources/DESIGN.md` and `unitTests/resources/query-array-scoping.test.js` plus the tracking epic #2440 — **documentation and regression scaffolding only, no production code**. Its fixture declares `{ name: 'sizes', elements: { type: 'Int' } }` with no top-level `type` key, which is exactly the programmatic route that hits `case undefined: return autoCast(value)` and therefore works. That is why the new pins do not show this defect.

#2440's divergence table (rows 1-14, "as of harper main, 2026-09") lists #2433 at row 10 and #2434 at row 11. **This mechanism — a single, unchained condition on a GraphQL-declared array attribute — is not a row in that table**, so it is currently tracked nowhere.

Contributor guide

Open the contributing guide

Research direction

Start by running integrationTests/qa-scratch/qa922-elements-coercion.test.ts and inspect resources/Table.ts around prepareConditions and coerceType, then compare the GraphQL array declaration in resources/graphql.ts. Update the regression spec at state/promote-candidates/P-651/qa922-elements-coercion.test.ts so it asserts the expected 25 records, and confirm the untyped numeric array queries pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
graphql, javascript
Domain
api, database
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.