oneapi-src / oneapi-src/unified-memory-framework

Add TLS cache to tracking provider/critnib

Open
#1,097 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C
Stars
98
Forks
48
Avg merge
3d 20h
Merged PRs (30d)
6

Description

At this moment, if a user wants to free an allocation from the proxy pool, they can do something like this:

umf_pool_by_ptr(ptr, &pool);
umf_pool_free(pool, ptr);

Internally, proxy_free() calls umfMemoryTrackerGetAllocInfo(ptr, &allocInfo); to retrieve the allocation size. Then, the free function from the tracking provider is called, which removes the entry for ptr from the critnib tree (the removal process itself involves a tree search).

In this example, the same pointer is being searched for three times (noting that the removal process also performs a tree search).

This issue proposes introducing caching of the last search in the tracking provider. We have two approaches:

Extend Critnib with a Caching Feature
This approach requires using functions like pthread_setspecific (though the performance impact of this function is uncertain).
With this method, the pointer lookup could be reduced to one search.

Add Cache at the Tracking Provider Level
Since the tracking provider is static, a TLS cache can be added easily using __thread.
However, with the current implementation of critnib, a tree search is still required for removal, so this approach would reduce the number of searches from three to two

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the tracking provider's proxy_free() path, the umfMemoryTrackerGetAllocInfo() call, and the critnib removal path described in the issue. Compare placing the TLS cache in critnib with placing it at the tracking-provider level; done means repeated pointer searches are reduced without changing allocation or free behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.