mafintosh / mafintosh/hyperdb

The definitive replication and authorization guide

Open
#153 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
748
Forks
56
PR merge metrics
No merged PRs in 30d

Description

I think we need to hash out and clarify the replication and authorization processes a bit. I have been struggling with this for many days now and based on issues here I'm not alone, so I'm hoping we can use this issue to clear things up.

After reading the documentation, reading the tests, going over the code and issues, I still can't get this right, so I think I'm not far off if I say that the whole replication process is not intuitive to implement. There are so many details to grok so let's go over it one step at a time. The biggest issue seems to be that it's not clear how the hyperdbs need to be set up for replication to work.

Here are two scenarios that I want to figure out, but it seems I can't.

**Scenario 1: I have hyperdb 1 that I want to read & write with hyperdb 2**

My logic:
1. Create 2nd hyperdb with 1st hyperdb's `local.key`
1. Authorize 2nd hyperdb to write 1st hyperdb
1. Create `replicate()` streams from both hyperdbs
1. Create socket connection between machines and do `stream1st.pipe(socket)` and `socket.pipe(stream_2nd)`

That's the structure I got from the docs etc. but it doesn't seem to work. There's the issue with "first hypercore must be the same" which I guess means I have to create 2nd hyperdb with `hyperdb(storage, hyperdb_1st.local.key)`. With that, I can see the connection happening, but nothing gets replicated. What steps are missing here?

**Scenario 2: I want to replicate another hyperdb without writing**

My logic:
1. Create own hyperdb with remote hyperdb's `local.key`?
1. Create `replicate({ live: true })` stream for my hyperdb
1. Connect to remote hyperdb's socket
1. Do `socket.pipe(my_hyperdb_stream)`

With this I don't get any errors, I see data going over the socket, and I can see the data in the remote hyperdb, but nothing shows up in my hyperdb. It's a bit like the replication doesn't start for some reason.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the existing documentation with the referenced tests, code, and related issues, focusing on hyperdb setup, authorization, and replicate() streams. Document the complete steps for both read/write replication and read-only replication, including what is missing from the two scenarios and how to verify that data appears on both sides.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
databases, distributed-systems, documentation, security
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.