v2/*Selective* CAR creators without duplicate puts may not be traversing complex DAGs
- Dominant language
- Go
- Stars
- 175
- Forks
- 54
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 4
Description
We have this in the `LinkSystem` config for the selective CAR creation utilities in v2:
```
LinkVisitOnlyOnce: !opts.BlockstoreAllowDuplicatePuts
```
These two things should probably not be directly related. Duplicate puts should be stopped at the CAR write output, but traversal should be able to roam through the DAG without skips _when_ we're doing a non-exhaustive selector on complex DAGs.
i.e. if we have a non-exhaustive selector and a DAG that contains multiple interlinked blocks (the definition of "complex DAGs" here is a bit squishy because de-duping of UnixFS data will often mean a UnixFS DAG has this property too), the traversal may need to re-visit blocks under different selector conditions. But `LinkVisitOnlyOnce` is only intended for the case where we're absolutely sure that re-visits are pointless.
v0's `SelectiveCar` handles this itself by keeping track of CIDs written and not rewriting out the ones that it's already seen, but it still lets the traversal go over all blocks. But we allow a `TraverseLinksOnlyOnce` option to be passed in by the user where they know it's OK (e.g. when they're doing an ExploreAll). Perhaps we should follow that pattern here.
I think that we probably should be solving this in the traversal engine instead, making `LinkVisitOnlyOnce` redundant and deprecating it because our selectors are smart enough to know not to revisit where it doesn't need to.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.