FilOzone / FilOzone/dealbot

Avoid duplicating the full piece buffer when parsing sampled-retrieval CARs

Open Beginner friendly
#616 1 comment 0 reactions 0 assignees View on GitHub
enhancement good first issue help wanted ready-for-work
Dominant language
TypeScript
Stars
3
Forks
9
Avg merge
4d 14h
Merged PRs (30d)
6

Description

`readBlocks` in the sampled-retrieval piece validation wraps the incoming piece buffer in `new Uint8Array(pieceBytes)` before handing it to `CarReader.fromBytes`:

https://github.com/FilOzone/dealbot/blob/2452e7deeb177ff6e94383386458f2a914ebf679/apps/backend/src/retrieval-anon/piece-validation.service.ts#L192-L200

`pieceBytes` is a `Buffer`, which already is a `Uint8Array`, so that wrapper allocates a second full copy of the entire piece. Sampled pieces run up to ~500 MiB, so peak memory for one check is roughly 2x the piece size for no reason.

Fix: pass `pieceBytes` straight to `CarReader.fromBytes(pieceBytes)`. Same behavior, drops the redundant allocation.

This is pre-existing from #487 and untouched by the rename in #612, so tracking it on its own. The file moves to `apps/backend/src/sampled-retrieval/piece-validation.service.ts` once #612 lands.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in apps/backend/src/retrieval-anon/piece-validation.service.ts at readBlocks and inspect how the piece buffer is passed to CarReader.fromBytes. Make the allocation-free change described in the issue, then verify sampled-retrieval piece validation retains the same behavior and no longer duplicates the full buffer; account for the later path move to apps/backend/src/sampled-retrieval/piece-validation.service.ts.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, performance
Issue type
Refactor
Difficulty
1/5
Estimated time
Under an hour
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
86/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.