HarperFast / HarperFast/harper

Track config-source provenance on hdb_nodes / hdb_role for deterministic config-driven deletes

Open
#601 0 comments 0 reactions 1 assignee Claimed by @Ethan-Arrowood View on GitHub
area:clustering enhancement from-jira
Dominant language
JavaScript
Stars
89
Forks
10
Avg merge
2d 2h
Merged PRs (30d)
205

Description

When `hdb_nodes` and `hdb_role` records are generated from configuration (`harperdb-config.yaml` and `roles.yaml`), the records carry no indication of *which* source created them. As a result, if an entry is removed from config, we can't safely deduce that the corresponding record should also be removed (it might have been created via the operations API, which should survive).

## Ask

- Stamp config-sourced records with their provenance (e.g. a `_source` field with the path or origin marker).
- On config change, identify entries that were present in the prior config but are now removed; deterministically delete the corresponding records *only if* they have the matching config-source provenance.
- API-created records remain untouched by config-driven deletion.

## Cluster wrinkle (from the Jira description)

The configuration "exists through a cluster" — multiple files on multiple nodes that may not be in sync. So a removed entry on one node doesn't necessarily mean it's removed cluster-wide. The proposed mechanism:

- Compare timestamps of records vs. timestamps of configuration changes.
- Only treat an entry as "removed by config" if the config change is recent enough to credibly explain the absence (i.e. newer than the record's creation/last-update time).

## Acceptance criteria

- New config-sourced records carry source/provenance metadata.
- Config-driven removal works deterministically: removing an entry from config and reloading causes the matching record to be deleted *if* and *only if* it was config-sourced and the config change post-dates the record.
- API-created records are not affected by config reloads.

## Out of scope

- The equivalent for tables / `schema.graphql` — different enough to warrant its own ticket (per the Jira description).

---

Tracked in Jira: [CORE-2967](https://harperdb.atlassian.net/browse/CORE-2967)

🤖 Filed by Claude on behalf of Kris.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.