dgraph-io / dgraph-io/dgraph

perf(unique): verifyUniqueWithinMutation is O(N^2), dominates @unique cost on batched mutations

Open
#9,814 0 comments 0 reactions 1 assignee View on GitHub

@shiva-istari is already working on this.

Since Sep 3, 2026.

Dominant language
Go
Stars
21.8k
Forks
1.6k
Avg merge
2d 5h
Merged PRs (30d)
9

Description

What

@unique write cost grows quadratically with the number of unique-predicate edges in a single
mutation. At 16k edges in one mutation a write takes 9.7s against 142ms for the same mutation on a
plain @index(exact) predicate.

The cost sits entirely in verifyUniqueWithinMutation, not in the injected uniqueness queries.

Measurements

v25.3.8, one mutation, p: string @index(exact) . against p: string @unique @index(exact) .:

edges/mutation plain @unique ratio growth per doubling
500 7.1 ms 19.4 ms 2.7x
1,000 12.6 ms 60.4 ms 4.8x
2,000 24.0 ms 203.6 ms 8.5x
4,000 39.2 ms 554.5 ms 14.1x 2.91x
8,000 74.1 ms 2,242.7 ms 30.3x 4.04x
16,000 141.6 ms 9,762.4 ms 68.9x 4.35x

Plain grows about 2x per doubling. @unique converges on 4x, which is O(N^2).

Single-edge mutations are unaffected: 1.66 ms/mutation with @unique against 1.77 ms plain over 300
sequential inserts. The cost scales with edges-per-mutation rather than with total write volume or
how hot the predicate is.

Where the time goes

The server latency breakdown separates the two candidate costs, because addQueryIfUnique runs
during parsing and the duplicate scan runs during processing:

N phase plain @unique
8,000 parsing 1.1 ms 1.4 ms
8,000 processing 93.4 ms 2,249.4 ms

Parsing stays flat, so injecting one eq() query per edge is cheap. The per-pair constant confirms
the nested loop as the source: 64M pairs / 2,249 ms at N=8k and 256M pairs / 9,762 ms at N=16k both
work out to 35-38 ns per pair.

Cause

verifyUniqueWithinMutation in edgraph/server.go iterates qc.uniqueVars (a
map[uint64]uniquePredMeta) inside itself. Two things make it worse than the algorithm requires:

  • the inner loop starts at for j := range qc.uniqueVars rather than i+1, so every pair is
    compared twice
  • dql.TypeValFrom(pred2.ObjectValue) is called on every pair before any match is established

Nested map iteration also explains why the per-pair constant is 35 ns rather than a few.

Suggested fix

Hash (predicate, value) into a set and make a single linear pass, keeping the first-seen subject so
the existing error message is preserved. Starting j at i+1 is a free 2x if a smaller change is
wanted first.

Impact

Live loader's default --batch is 1000, which lands at the ~5x point and never reaches the bad part
of the curve. Application code that builds one large mutation from an import is where this bites.

Numbers are single-threaded HTTP against a dgraph/standalone:v25.3.8 container, so treat the
absolute values as indicative. The scaling exponent is the durable part, and it is CPU-bound work
that will contend across concurrent writers rather than disappear.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.