HarperFast / HarperFast/harper

Removing an @relationship on a populated table breaks select() over that field: transient phantom edges, then a permanent 500 (Resource.ts:867)

Open
#1,788 0 comments 0 reactions 0 assignees View on GitHub
area:clustering area:rest-api bug
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 6h
Merged PRs (30d)
200

Description

### Bug Summary

Removing an `@relationship` from a **populated** table's schema breaks `select()` over that field in two distinct ways that share one trigger: first a transient **data leak** (removed-relationship edges keep resolving), then a **permanent 500 crash** that survives a full process restart. There is no clean "the field no longer exists" behavior at any point — a `select()` referencing the removed relationship should return an absent/4xx result, never stale edges and never an unhandled 500.

Found via exploratory QA (adjudicated from a demoted schema-evolution test). No data loss — the full-scan path is unaffected.

### Steps to Reproduce

1. Deploy a component with two related tables and a `@relationship`, e.g. `Author.tagBooks: [Book] @relationship(to: tag)` (with `Book.tag: ID @indexed`), and populate it (e.g. 20 authors × 25 books).
2. Confirm `GET /Author/?select(tagBooks{id})` resolves the related books.
3. Evolve the component schema to **remove** the `@relationship` (drop `Author.tagBooks` and the `@relationship` directive), and apply it via `set_component_file` + `restart_service http_workers` (i.e. an in-place schema change without a full restart).
4. Immediately issue `GET /Author/?select(tagBooks{id})` again, repeatedly, on a **multi-worker** instance (`threads.count: 4`).

### Expected Behavior

After the relationship is removed, a `select(tagBooks{...})` referencing it should return a clean result — the field is absent (empty/omitted) or a 4xx — deterministically, on all workers.

### Actual Behavior

Two separable bugs, one trigger:

**Leg A — transient phantom edges (stale-data exposure).** Immediately after the schema change is applied via `restart_service http_workers`, stale in-memory relationship-resolver metadata still resolves **all** the removed relationship's edges at HTTP 200. In single-worker mode the window is narrow (~1s). **In 4-worker mode it persisted across every request in the test with no decay** — effectively indefinite under an http-worker-only restart, since the workers hold stale resolver state. Clients receive data for a relationship that no longer exists in the schema.

**Leg B — permanent crash (deterministic).** The resolver state then flips to a hard failure:

```
TypeError: Cannot read properties of undefined (reading 'propertyResolvers')
at transformForSelect (resources/Resource.ts:867)
```

`transformForSelect(select, resource)` does `const propertyResolvers = resource.propertyResolvers`, but `resource` (the removed relationship's target resource) is now `undefined`. Every subsequent request to that `select` returns **500**, **including after a full process restart** (`killHarper` + `startHarper` on the same `dataRootDir`). Waiting never converges to a correct state; a full restart only swaps Leg A → Leg B.

### Blast radius / notes

- Affects REST reverse-expansion and GraphQL nested `select` over the removed field. `search_by_conditions` (full scan) **never regressed — no data loss.**
- Reproduced ×3, deterministic (not flaky). **No storage-engine sensitivity** — identical on RocksDB and LMDB. Worker count only affects how long the Leg A leak window lasts, not the eventual broken state.
- The adjacent case of removing an `@indexed` annotation (without a relationship) is **not** a defect — it correctly falls back to a full scan (the underlying attribute data was never removed).

### Suggested fix direction

1. Invalidate / rebuild the per-worker relationship-resolver metadata when an `@relationship` is removed (Leg A — the workers should not keep resolving a dropped relationship).
2. Guard `transformForSelect` against an `undefined` target resource so a `select()` over a removed relationship yields a clean absent/4xx result instead of an unhandled 500 (Leg B).

Likely related to the relationship/index family (#1485).

### Environment

- harper `main` — confirmed on `7d10daf7d`; the `resources/Resource.ts` / `resources/tables/Table.ts` relationship-resolver path is unchanged through current `56f8891b9` (v5.2.0-alpha.4), so it reproduces on current main.
- A local integration repro exists (`integrationTests/qa-scratch/qa-phantom-edge-adjudicate.test.ts`) exercising both legs across rocksdb + lmdb and single/multi-worker; it can be promoted as a regression anchor once fixed.

Filed from exploratory QA · characterized with Claude (Fable 5)

Contributor guide

Open the contributing guide

Research direction

Start with transformForSelect in resources/Resource.ts and the relationship-resolver path in resources/tables/Table.ts. Run integrationTests/qa-scratch/qa-phantom-edge-adjudicate.test.ts across RocksDB and LMDB with single and multi-worker setups. Done means removing a relationship invalidates stale resolver metadata and a subsequent select returns an absent or 4xx result rather than stale edges or a 500.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.