openvinotoolkit / openvinotoolkit/npu_compiler
Multi-Tenant Behavior and Resource Sharing on Intel NPU4
Nobody has claimed this yet.
- Dominant language
- MLIR
- Stars
- 100
- Forks
- 50
- Avg merge
- 3h 57m
- Merged PRs (30d)
- 1
Description
Hi, I'm profiling workloads on the Intel NPU4 architecture and have some questions regarding multi-tenant usage.
The manual mentions 6 tiles with corresponding CMX. My main concern is how different users or processes share the NPU. Specifically:
Can multiple users/processes utilize different NPU tiles concurrently? Or is the NPU shared via time-division multiplexing?
If concurrent tile usage is possible, how is the SHAVE L2 Cache shared among them?
Is the scheduling of different users handled by hardware or software?
Understanding these aspects is crucial for optimizing our workload deployment. Any insights would be greatly appreciated.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the Intel NPU4 manual sections covering its six tiles, CMX, SHAVE L2 Cache, and scheduling. Determine whether the documented behavior answers the questions about concurrent process usage, cache sharing, and hardware versus software scheduling. Done means providing authoritative answers or identifying the missing documentation.
Written by the indexing model from the issue text.
Assessment
- Domain
- compilers
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100