iree-org / iree-org/iree

Revive C-based file mapping API and be smart about it

Open
#3,909 2 comments 0 reactions 1 assignee Claimed by @benvanik View on GitHub
performance ⚡ runtime runtime/api
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

This used to be part of the C API but I dropped it in favor of waiting util we had a bit more usage (and a cleaner API) before exposing it. https://github.com/google/iree/blob/08237ac1c7bec3435cd7c626802dfad1b91bbd0e/iree/base/api.h#L462-L488

We should do this again but this time add some of the functionality that lets us abstract over mapped and unmapped memory. Something like iree_memory_buffer_t or something that could either be a mmapped file, shared memory (shm/etc), or just a normal range of bytes.

Features
* [ ] page size support query (ideal page size, page granularity) and large pages
* [ ] nocache/streaming
* [ ] flush
* [ ] locking (wire to physical pages like VirtualLock)
* [ ] discard (DiscardVirtualMemory for writes) or offer (OfferVirtualMemory for reads)
* [ ] subrange prefetching (madvice/PrefetchVirtualMemory/touch striding for malloc)
* [ ] protection (read/readwrite/execute/etc) - useful for JIT/sharing
* [ ] interface + implementations:
* [ ] allocate buffer and manage lifetime with iree_allocator_t (possibly wrap, etc)
* [ ] circular buffer address space overlays: see [1](https://abhinavag.medium.com/a-fast-circular-ring-buffer-4d102ef4d4a3) and [2](https://lo.calho.st/posts/black-magic-buffer/)
* [ ] file mapping:
* [ ] from fd/HANDLE/path

Unblocks better filesystem-less dylib loading (#3845) and enclave explorations (#3910).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.