`xnn_weights_cache_provider` look_up doesn't work?
- Dominant language
- C
- Stars
- 2.5k
- Forks
- 560
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 163
Description
I've recently been experimenting with XNNPACK's weight cache to reduce load time by caching packed weights and also reduce memory pressure for repeated weights across the same kernels.
I was experiementing with fully-connected operator and found that the weight cache was never being hit. I noticed that when using the apis to create the `xnn_weights_cache_t` we set the look up function to be `xnn_internal_weights_cache_look_up`:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/runtime.c#L148
looking at this function, it looks like a placeholder function which would always return `XNN_CACHE_NOT_FOUND`:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/cache.c#L491-L496
Now when I'm using the weights cache to create a runtime_t with only a fully connected operator, in the flow of creating the fully-connected operator, we look up the cache to see if the weights have been packed before, using xnn_weights_cache_look_up:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/operators/fully-connected-nc.c#L154-L157
However this just uses the the placeholder function above, returning XNN_CACHE_NOT_FOUND:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/cache.c#L530-L534
As a result, every look up would then fall to XNN_CACHE_NOT_FOUND, in which weights have to be repacked, and memory has to be allocated for the newly packed weights:
https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/operators/fully-connected-nc.c#L159-L179
Am I looking at this incorrectly? Or is this a feature that is still a wip? Or is this a bug that is meant to be fixed in the future?
Contributor guide
Assessment
This issue has not been assessed yet.