ARK-Builders / ARK-Builders/ark-core

Representative resources and dedicated events for duplicates/collisions

Open
#93 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
10
Forks
6
PR merge metrics
No merged PRs in 30d

Description

We might need separate events to track duplicates. Something like `DuplicateAdded(id, path)` and `DuplicateRemoved(id, path)`. Although, I'm not sure that duplicate removal can be useful, maybe duplicate addition is enough. It could be used to allow the user to select representative manually. Just idea for future.

We could simplify `added` field of the `IndexUpdate` structure. From API point of view, we don't need a collection of paths attached to the addition event, only one path (representative). We can take any path as a representative of the group, because we don't distinguish duplicates. A single representative should be enough for the app to do something with it, e.g. render thumbnail.

So when unique resource is detected, we take its path as the representative. When a duplicate appears, we skip it or emit its path in separate event. If during unique addition, several paths were introduced at once, we take the shortest path and use it as representative. All other paths should be mentioned in `DuplicateAdded` event.

----

We can also use term "Collision" since in case of non-cryptographic hash function distinct bytes can result in same id.

Contributor guide

Open the contributing guide

Research direction

Start by locating the IndexUpdate structure and the code that emits resource-addition events. Clarify whether representatives alone or separate duplicate/collision events are required, then define the event behavior for unique resources, duplicates, and multiple paths before implementing and validating the API changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.