holepunchto / holepunchto/hyperbee2

How should truncate() of local core be handled?

Open
#30 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
12
Forks
5
Avg merge
3h 23m
Merged PRs (30d)
27

Description

Calling truncate() on the underlying Hypercore seems to cause a few issues:

It can break get() and iteration (and probably other things):

import Hyperbee from './index.js'
import Corestore from 'corestore'

{
    const b = new Hyperbee(new Corestore('./sandbox/store'))
    await b.ready()

    const w = b.write()
    w.tryPut(Buffer.from('1'), Buffer.from('1'))
    await w.flush()

    const w2 = b.write()
    w2.tryPut(Buffer.from('2'), Buffer.from('2'))
    await w2.flush()

    await b.close();
}

{
    const b = new Hyperbee(new Corestore('./sandbox/store'))
    await b.ready()
    await b.core.truncate(1);
    // Does not complete:
    console.log((await b.get(Buffer.from('1'))).value.toString());
    // Never called:
    console.log('done');
    await b.close();
}

Will result in:

Warning: Detected unsettled top-level await at file:///...

But opening the store again after the truncate, it will work again (without key=2 in this case). The unsettled top-level await
it a particularly painful failure mode as you can't catch it with a regular try/catch around the block. The program may just stop running.

Truncating the core also does not emit an 'update' event on the Hyperbee instance when autoUpdate is true. That might be intentional, but if a user re-opens the store there will have been a change without any corresponding update event:

import Hyperbee from './index.js'
import Corestore from 'corestore'

{
    console.log('-- write --');
    const b = new Hyperbee(new Corestore('./sandbox/store', {autoUpdate: true}))
    b.on('update', () => console.log('update'));
    await b.ready()

    const w = b.write()
    w.tryPut(Buffer.from('1'), Buffer.from('1'))
    await w.flush()

    const w2 = b.write()
    w2.tryPut(Buffer.from('2'), Buffer.from('2'))
    await w2.flush()

    console.log((await b.get(Buffer.from('1')))?.value?.toString());
    console.log((await b.get(Buffer.from('2')))?.value?.toString());
    await b.close();
}

{
    console.log('-- truncate --');
    const b = new Hyperbee(new Corestore('./sandbox/store'), {autoUpdate: true})
    b.on('update', () => console.log('update'));
    await b.ready()
    await b.core.truncate(1);
    await b.close();
}

{
    console.log('-- re-open --');
    const b = new Hyperbee(new Corestore('./sandbox/store'), {autoUpdate: true})
    b.on('update', () => console.log('update'));
    await b.ready()
    console.log((await b.get(Buffer.from('1')))?.value?.toString());
    console.log((await b.get(Buffer.from('2')))?.value?.toString());
    await b.close();
}

Will output:

-- write --
update
update
1
2
-- truncate --
-- re-open --
1
undefined

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reproductions in the issue and inspect index.js around Hyperbee initialization, b.core.truncate(1), get(), iteration, and the autoUpdate update event. Establish the intended behavior after truncation, including whether reads complete and whether an update is emitted; the issue is resolved when those behaviors are defined and covered by a reliable reproduction or regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.