payloadcms / payloadcms/payload
drizzle: negated operators (not_in, not_equals, not_like) on to-many paths use ANY-row semantics instead of NO-row (should compile to NOT EXISTS)
@r1tsuu is already working on this.
Since Aug 17, 2026.
- Dominant language
- TypeScript
- Stars
- 44.8k
- Forks
- 4.2k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 53
Description
Describe the Bug
On the SQL adapters, negated operators (not_in, not_equals, not_like) against a sub-field reached through a to-many path — an array field's sub-field, or a hasMany relationship — match a document when any one of its rows fails the positive condition, instead of when no row matches it.
Since a to-many field with 2+ rows virtually always has some row that differs from the queried value, these clauses degrade to matching (nearly) every document. db-mongodb has the correct semantics for the same query ($nin/$ne against an array match only when no element matches), so the adapters silently disagree.
Mechanism, from current main (packages/drizzle/src/queries):
- When a where path traverses an array or hasMany relationship,
getTableColumnFromPath.tspushes a LEFT JOIN to the rows table (eq(parent.id, subTable._parentID), markedisOneToMany: true) and resolves the path to a column on the joined table. parseParams.tsapplies the operator constraint directly to that joined column in the flat WHERE — identically for every operator, with no awareness that the column is on the "many" side of a join.selectDistinct.tsdedupes the row fan-out.
That architecture gives EXISTS semantics ("some row matches") to every operator. For positive operators (equals, in, like, contains) that is the desired meaning on a to-many path. For negated operators it is wrong: per-row negation yields "at least one row differs" where the only useful meaning is "no row matches".
Concrete example (the nested-docs case from payloadcms/payload#17658): every doc's breadcrumbs array contains a self-row plus ancestor rows, so
where[breadcrumbs.doc][not_in][0]=391
compiles to roughly
SELECT DISTINCT c.id
FROM categories c
LEFT JOIN categories_breadcrumbs br ON br._parent_id = c.id
WHERE br.doc_id NOT IN (391) -- per-row: true for any non-391 breadcrumb row
which matches every document (including all descendants of 391, each of which plainly carries a doc: 391 breadcrumb row in the same response). What it should compile to is the De Morgan form:
SELECT c.id
FROM categories c
WHERE NOT EXISTS (
SELECT 1 FROM categories_breadcrumbs br
WHERE br._parent_id = c.id
AND br.doc_id IN (391)
)
The same applies to not_equals and not_like on any array sub-field or hasMany relationship, on db-postgres, db-vercel-postgres, and db-sqlite alike (shared packages/drizzle query builder).
Reproduction Steps
No custom code needed beyond an array field:
- Any SQL adapter. A collection
postswith an array fieldtags: [{ name: 'tag', type: 'text' }]. - Create doc A with tags
["x", "y"], doc B with tags["y"], doc C with no tags. - Query
where[tags.tag][not_equals]=x(orwhere[tags.tag][not_in][0]=x).
Expected (and what db-mongodb returns for the equivalent data): B and C — documents where no tag is x.
Actual on SQL adapters: A, B — A matches because its y row satisfies the per-row <> 'x', and C is dropped because the LEFT JOIN produces a NULL row that fails the constraint. Both halves are wrong, in opposite directions.
The NOT EXISTS form fixes both at once: A is excluded (a row matches x), and C matches naturally (no rows at all), without the ad-hoc IS NULL OR ... wrappers parseParams.ts currently applies only to flat columns.
Impact / prior reports
- payloadcms/payload#17658 —
plugin-nested-docs' defaultparentFilterOptionsrelies on'breadcrumbs.doc': { not_in: [id] }, which this bug makes completely inert on Postgres: the parent picker offers the doc's own descendants, editors create circular references from the default UI, and the save then hangs in the plugin's unguarded ancestor walk (#16517). - #5248 (2024) —
filterOptionswithnot_inreturning wrong results on Postgres once a relationship has a value; closed without a fix, reporter confirmed it still reproduced, now locked. Consistent with this root cause. - PR #17769 works around the nested-docs symptom by resolving descendant IDs with a positive lookup and excluding by plain
id: { not_in }— the right call at the plugin level precisely because it stops depending on these semantics. But the adapter bug remains for every user-written query orfilterOptionsusing a negated operator on a to-many path, and nothing errors — queries just silently return wrong result sets.
Suggested direction
When the operator is not_in/not_equals/not_like and the resolved path crossed one or more isOneToMany joins, compile the positive form of the constraint into a correlated subquery over the joined table chain (the correlation predicate already exists as the join's condition) and wrap it in Drizzle's notExists(), instead of registering the join on the outer query.
Known complications, for whoever picks this up:
getTableColumnFromPatheagerly mutates the sharedjoinsarray, so the join(s) added for a negated path need to be relocated into the subquery — but only when no other operator on the same path in the same where clause still needs the outer join.- Needs to compose with nested arrays (chained joins), localized sub-tables (
_localeconditions), hasMany relationships via the_relstable, blocks, and the polymorphic-relationship branch inparseParams.ts. - It is a silent behavior change for any query currently compensating for the broken semantics, so it wants regression coverage across all three SQL adapters and probably a release note.
Environment Info
payload: 3.80.0
@payloadcms/db-postgres: 3.80.0
next: 16.2.1
Node: 22.22.1
Database: PostgreSQL 18
Reproduces on current main — the operator handling in packages/drizzle/src/queries/parseParams.ts has no to-many awareness for negated operators, and notExists is unused anywhere in packages/drizzle.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.