CodeForPhilly / CodeForPhilly/jawn

jawn as a wrapper around levelup and hypercore

Open
#15 8 comments 0 reactions 0 assignees View on GitHub
discussions index/db
Dominant language
JavaScript
Stars
46
Forks
8
PR merge metrics
No merged PRs in 30d

Description

@ogd @mafintosh as I spec out the basic features for dat-tables, it looks like our hypercore feeds will basically contain a log of CREATE, UPDATE, and DELETE operations. I want to avoid writing an API that unnecessarily duplicates the leveldown API or (gadzooks) ending up with yet another SQL-ish API that nobody wanted. Would it be easier to make levelup more central in the operations and implement a sort of 'smart' log that follows 'put' and 'delete' events emitted by levelup, writing the changes to a hypercore feed?
## Assumptions

The key features for an MVP of dat-tables are:
- provide an api for importing & exporting data
- write imported data to a hypercore feed
- track those data in an index (for lookups and in order to distinguish between 'adds' and 'updates')

I had been thinking of the index as a tail of the feed -- always reading operations from the feed (even locally) and applying them to the index accordingly -- but I think that was the wrong approach.
## dat-tables as a wrapper around levelup and hypercore

_Note: The levelup database always represents the HEAD of your data._
- **Wrap levelup**: Provide an import/export api that's a transparent wrapper around the levelup api
- **Log levelup events to hypercore**: During `import` operations, follow 'put' and 'delete' events emitted by levelup. Write all changes to a hypercore feed (ignore `put`s that didn't change anything)
- **Tail the hypercore feed when syncing**: When syncing changes from other hypercore hosts, apply those changes from the feed directly to levelup

This lets us focus on making dat-tables support operations like 'checking out' old versions and running diffs against old versions of the data. It also lets us focus on refining the API to support target use cases.

Does this make sense? Am I missing anything?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.