google / google/XNNPACK

`xnn_weights_cache_provider` look_up doesn't work?

Open
#6,257 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
2.5k
Forks
560
Avg merge
1d 6h
Merged PRs (30d)
163

Description

I've recently been experimenting with XNNPACK's weight cache to reduce load time by caching packed weights and also reduce memory pressure for repeated weights across the same kernels.

I was experiementing with fully-connected operator and found that the weight cache was never being hit. I noticed that when using the apis to create the `xnn_weights_cache_t` we set the look up function to be `xnn_internal_weights_cache_look_up`:

https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/runtime.c#L148

looking at this function, it looks like a placeholder function which would always return `XNN_CACHE_NOT_FOUND`:

https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/cache.c#L491-L496

Now when I'm using the weights cache to create a runtime_t with only a fully connected operator, in the flow of creating the fully-connected operator, we look up the cache to see if the weights have been packed before, using xnn_weights_cache_look_up:

https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/operators/fully-connected-nc.c#L154-L157

However this just uses the the placeholder function above, returning XNN_CACHE_NOT_FOUND:

https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/cache.c#L530-L534

As a result, every look up would then fall to XNN_CACHE_NOT_FOUND, in which weights have to be repacked, and memory has to be allocated for the newly packed weights:

https://github.com/google/XNNPACK/blob/85071b8b8729f63262484942fc9eb1c7c16525c4/src/operators/fully-connected-nc.c#L159-L179

Am I looking at this incorrectly? Or is this a feature that is still a wip? Or is this a bug that is meant to be fixed in the future?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.