Indexes are slurpy and allocates lots of memory
- Dominant language
- Go
- Stars
- 175
- Forks
- 54
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 4
Description
Indexes datastructure are fully copied into memory when parsing an index.
There is no limit on how big thoses copies were (because practical indexes used by peoples are bigger than what would be a safe limit).
So if you parse an index from untrusted user input, they can send you a really big index and memory hog or OOM you.
Would be nice if we have a way to parse untrusted indexes without opening yourself to OOMs.
This was first attempted in #312 (see discussion in https://github.com/ipld/go-car-priv/pull/2):
- https://github.com/ipld/go-car/commit/3923d315f1083ce25ee4e790287cc5702a381b13
But was reverted due to API breaking considerations (see discussion in the channel and https://github.com/ipld/go-car-priv/pull/18)):
- https://github.com/ipld/go-car/commit/4bc677484af39d1153e075ae162c5ebcc6e396a3
See https://github.com/ipld/go-car/security/advisories/GHSA-9x4h-8wgm-8xfg
> *go-car@v2.4.0* also includes additional documentation regarding the dangers of consuming CARv2 index data from untrusted sources and a recommendation to regenerate indexes of CAR data from such sources where an index is required.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.