max-mapper / max-mapper/dat-core

dat-core v5 API

Open
#34 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
42
Forks
6
PR merge metrics
No merged PRs in 30d

Description

we wanna strip down the dat-core API to make it simpler/reduce scope.

here's the proposed new dat-core features:
- replication
- change processor hooks
- low level graph data manipulation api (similar to hyperlog api)
- graph external data pointers (with hashes pointing to commits and blobs etc)
- encoding support (abstract-encoder)
- core should be pure JS (works in browser and everywhere)

the goal is to be able to use dat-core in a situation where we might currently use hyperlog, e.g. https://github.com/moose-team/thanks/blob/fcfac4f7f4581f0fa0fecfe90f2122445da22ff4/index.js. dat-core is currently too heavy for these kinds of custom use cases. being able to use it for hyperlog-style things will let us build custom things while still being compatible with the entire dat high level ecosystem (like replication etc)

this means we would remove all other existing functionality currently in this repo and make new modules that the dat cli can use directly (tentative names):
- dat-fs (for blobs)
- dat-datasets (for tables and key/value storage)
- dat-containers (for VMs)
- dat-blob-resolver
- dat-commit-resolver

the dat cli won't necessarily change because of any of this, this is just a technical refactoring

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No specific files or tests are named. Start by reviewing the current dat-core modules and the proposed dat-fs, dat-datasets, dat-containers, dat-blob-resolver, and dat-commit-resolver boundaries, then compare how the dat CLI uses them. Done would require an agreed smaller API and a refactoring that preserves replication and compatibility with the dat ecosystem.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.