Level / Level/community

plugins v2 (sectioned db, with hooks)

Open
#53 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

discussion
Dominant language
No language data
Stars
51
Forks
15
PR merge metrics
No merged PRs in 30d

Description

Basically, instead of hooking new behaviour into the main database, you can create sub sections:

SubLevel(db)
var fooDb = db.sublevel('foo', '~')

fooDb is an object with the levelup api, except when you do an fooDb.put('bar') the key is prefixed with ~foo~ so that it's separated from the other keys in the main db. This is great, because if you want to build a section of the database with special behaviour in it, you can create a subsection, and extend the behaviour in anyway you like -- but it will only affect that section! So, you can monkeypatch it - whatever, you won't introduce bugs into other parts of the program.

Most of the plugins I built needed some sort of interception, where a value is inserted into one range, which triggers an action which inserts something into a different section. To get reliably consistent data this needs to be atomic.

so, you can tie set hooks on a subsection, that trigger an insert into another subsection.

when a key is inserted into the main db, index that write with a timestamp, saved into another subsection.

var SubLevel = require('level-sublevel'); SubLevel(db)
var sub = db.sublevel('SEQ')

db.pre(function (ch, add) {
  add({
    key: ''+Date.now(), 
    value: ch.key, 
    type: 'put'
  }, sub) //NOTE pass the destination db to add
          //and the value will end up in that subsection!
})

db.put('key', 'VALUE', function (err) {

  //read all the records inserted by the hook!
  sub.createReadStream()
    .on('data', console.log)
})

db.pre(function hook (ch, add) {...}) registers a function to be called whenever a key is inserted into that section. ch is a change, like a row argument to db.batch: {key: k, value: v, type: 'put' || 'del'}.

If the hook function calls add(ch) ch is added to the batch (a regular put/del is turned into a batch) but, if a subsection is passed in add(ch, sub) then that put will be added to that subsection.

Compare subsection code for queue/trigger https://github.com/dominictarr/level-sublevel/blob/e2d27cc8e8356cde6ecf4d50c980c2ba93d87b95/examples/queue.js
with the old code -
https://github.com/dominictarr/level-queue/blob/master/index.js
and https://github.com/dominictarr/level-trigger/blob/18d0a1daa21aab1cbc1d0f7ff3690b91c1e0291d/index.js

the new version is only ~ 60 lines, down from about ~ 200, also it's possible to use multiple different queue/trigger libs within the same db. And also, there is no tricky code that refers to ranges or prefixes in the subsection based code!

in summary,

  • create subsections, add any features to your subsection.
  • use pre(fun) to trigger atomic inserts into your subsection.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing level-sublevel/examples/queue.js with level-queue/index.js and level-trigger/index.js, as linked in the issue. Treat the proposal as design work: completion would require subsections and pre hooks that support atomic inserts into other subsections, with behavior verified for the described LevelUP-style operations.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.