Revive C-based file mapping API and be smart about it
- Dominant language
- C++
- Stars
- 3.9k
- Forks
- 1k
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 47
Description
This used to be part of the C API but I dropped it in favor of waiting util we had a bit more usage (and a cleaner API) before exposing it. https://github.com/google/iree/blob/08237ac1c7bec3435cd7c626802dfad1b91bbd0e/iree/base/api.h#L462-L488
We should do this again but this time add some of the functionality that lets us abstract over mapped and unmapped memory. Something like iree_memory_buffer_t or something that could either be a mmapped file, shared memory (shm/etc), or just a normal range of bytes.
Features
* [ ] page size support query (ideal page size, page granularity) and large pages
* [ ] nocache/streaming
* [ ] flush
* [ ] locking (wire to physical pages like VirtualLock)
* [ ] discard (DiscardVirtualMemory for writes) or offer (OfferVirtualMemory for reads)
* [ ] subrange prefetching (madvice/PrefetchVirtualMemory/touch striding for malloc)
* [ ] protection (read/readwrite/execute/etc) - useful for JIT/sharing
* [ ] interface + implementations:
* [ ] allocate buffer and manage lifetime with iree_allocator_t (possibly wrap, etc)
* [ ] circular buffer address space overlays: see [1](https://abhinavag.medium.com/a-fast-circular-ring-buffer-4d102ef4d4a3) and [2](https://lo.calho.st/posts/black-magic-buffer/)
* [ ] file mapping:
* [ ] from fd/HANDLE/path
Unblocks better filesystem-less dylib loading (#3845) and enclave explorations (#3910).
Contributor guide
Assessment
This issue has not been assessed yet.