ceramicnetwork / ceramicnetwork/.github

Streams do not support atomic operations

Open
#73 8 comments 1 reaction 0 assignees View on GitHub
bug dontclose help wanted
Dominant language
No language data
Stars
362
Forks
35
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
When you have a stream with constant read/write* operations on top of it by the same key, a race condition can occur when trying to write data on the stream that relies on the last read. In short, the node does not provide [locking](https://en.wikipedia.org/wiki/Read%E2%80%93modify%E2%80%93write) abilities into the stream per read, and as a result, the latest write on the stream is considered the latest commit, and the latest state to show for the stream when using `load` by any ceramic client. This can create a mismatch on the data the stream has. The problem only becomes more likely when the stream grows in size (e.g. `25 KB`)

\* _By read/write we refer to executing a `load` operation on our node, followed by an `update` operation in <100 ms_

**To Reproduce**
Steps to reproduce the behavior:
1. Create a stream as a JSON map, ideally in `25kbs` in size
2. Create an async script that maps over `[1...100]` and a) load the stream, b) updates the content, c) updates the stream
3. See the stream latest state being random every time, instead of being sequential.

**Expected behavior**
Ideally, we can load a stream with a `lock` identifier, where it can not be written by a write operation that doesn't have that identifier as part of its read. E.g.

```
const doc = await TileDocument.create(client, null, {
deterministic: true,
tags: ["hopr-dashboard"],
});

const mutatedDoc = Object.assign({}, doc.content, {
"foo": "bar,
});

doc.update(mutatedDoc, doc.lockId);
```

This behavior should be considered the default for all the ceramic clients, and invisible for users. If the lock identifier is not provided, then the Ceramic node should return a [428 HTTP status code error](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/428). To avoid infinite locking, streams should have a `60 secs` timeout, option which could be configured per stream.

**Screenshots**
Using [Documint](http://documint.net/k2t6wyfsu4pg1bz6houhqzlpljag79xcs6r04s1ihyapjhwzg8fl3e7fwrb1pg), you can see that `k2t6wyfsu4pg1bz6houhqzlpljag79xcs6r04s1ihyapjhwzg8fl3e7fwrb1pg` commit changes can have negative deltas on the content of the `next` state, where it should only have positive deltas (i.e. only additions to the content). In other words, the stream should only increase in size, not decrease.

**Ceramic versions**
We saw this issue in the [Clay](https://ceramic-clay.3boxlabs.com/api/v0/node/healthcheck?) node during the days of Sept 3rd and 10th of 2021.

**Machine, OS, browser information (please complete the following information):**
The client is loaded with the `key-did-provider-25519` and written using a serverless Vercel instance with node.js `14`. The code is executed [here](https://github.com/hoprnet/hopr-leaderboard/blob/098cb7c7d266172053143bc29c0b32e32b5f766c/pages/api/sign/verify/%5Baddress%5D/dashboard/%5BstreamId%5D.js#L56-L67).

**Additional context**
We are using a specific stream (`k2t6wyfsu4pg1bz6houhqzlpljag79xcs6r04s1ihyapjhwzg8fl3e7fwrb1pg`) as a "database" for keeping records of all other streams in the network. This stream is being written by [this](https://github.com/hoprnet/hopr-leaderboard/blob/098cb7c7d266172053143bc29c0b32e32b5f766c/pages/api/sign/verify/%5Baddress%5D/dashboard/%5BstreamId%5D.js#L56-L67) endpoint, triggered on the server-side by a server-side key. As a result, given the conditions for the requests are met, this read-modify-write call can be done within <100ms. The goal was meant to use this stream as an "indexer" of sort for our [Dashboard](https://network.hoprnet.org/), to keep track of all the streams our users where creating (with the same key).

Without this ability, streams are only limited to be written by a single user and single key, which although is good enough for some cases, seems limited to multiple users and single key cases, managed by servers. Indexing is one of the hardest problems in the decentralized ecosystem and would be great to allow Ceramic nodes to provide a solution by ensuring streams execute atomic operations on top of them.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.