Add per-method instance workspace option
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5k
- Forks
- 1.2k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 581
Description
In the XNNPACK backend, we may want a per-method instance workspace sharing option. This would be a good default option that would not introduce unwanted synchronization / serialized execution.
With this mode, memory would be shared between delegate calls in each copy of a loaded method.
Memory would not be shared in the following cases:
- Between multiple methods in one model - encode / decode, for example.
- Between multiple loaded copies of a single method - such as when loading a method to run simultaneously on multiple threads.
- Between separate PTEs.
This option will require some sort of unique method identifier, accessible in the backend interface. We have the method name, but this does not allow us to distinguish between multiple instances of the same method.
cc @digantdesai @mcr229 @cbilgin
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the XNNPACK backend interface and trace how method names and workspace state are represented. Determine where a unique per-method-instance identifier and the stated sharing boundaries would be specified. Done means the backend can distinguish the listed method instances without introducing unwanted synchronization or sharing across those boundaries.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100