mafintosh / mafintosh/hyperdb

How to efficiently get all heads *prior* to a node?

Open
#48 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
748
Forks
56
PR merge metrics
No merged PRs in 30d

Description

When building a [hyperdb-index](https://github.com/noffle/hyperdb-index), a common desire is to, when processing a node, to remove all previous heads from the index for that key, and replace them with the new value.

With hyperlog, `node.links` gave those hashes. How could this be done with hyperdb? I hacked it together in a roundabout way:

1. modify the new node's `node.clock` back one entry for its local feed
2. convert `node.clock` into a buffer of heads compatible with `hyperdb#version`
3. do a `hyperdb#checkout` of that version (one entry before this new node was inserted)
4. do a lookup on `node.key`: these are the old heads

This works, but it's a pretty heavy process, especially in the tight loop of an indexer.

@mafintosh can this be done more efficiency with the already existing incremental index?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the incremental index behavior and the existing hyperdb#version and hyperdb#checkout entry points described in the issue. Trace how node.clock and node.key are used in the workaround, then determine whether the incremental index can expose the prior heads efficiently. Done means the project has a decided, documented approach or an implemented API for retrieving those heads.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.