AccelerateHS / AccelerateHS/accelerate

Improved memory management

オープン
#240 コメント 9 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Haskell
スター
1k
フォーク
135
PR マージ指標
30日以内にマージされた PR はありません

説明

I've created this bug so we can discuss and track work on better memory management in Accelerate.

The status quo:
- The accelerate CUDA backend maintains a table that matches host side arrays to device arrays.
- A second "nursery" table is also maintained. It matches array sizes to device arrays.
- When device arrays are "freed" `cudaFree` is not actually called on the underlying device memory, rather an entry is added to the nursery table.
- When a host side array does not have a corresponding device array, a lookup of the nursery is performed to see if an array of the right size is already available for use.
- Device arrays are only ever actually `cudaFree`d when `cudaMalloc` fails and the nursery is flushed.

The advantage of the current approach is it minimises allocations by use of the nursery. One disadvantage is that we are heavily dependent on GHCs GC to know when arrays are no longer needed. As is evident in #165, this can cause a lot of problems. Additionally, there are cases where an array is still reachable from the perspective of the GC, but we know it's never going to be needed again. This is a problem when using the new streaming interface to process more data than will fit in device memory. Lastly, and perhaps less significantly, we have a large memory footprint, which rather assumes Accelerate is the only thing using the device memory.

A related issue is that a lot of the logic for memory management is the same, or similar, for different backends and it would make sense to move this into the Accelerate base package and provide an interface backends can use.

I know that you had some ideas about this @tmcdonell and @rrnewton, but I've written a simple interface below to show my current thinking. The more I think about this, the more I see that there are many things we need to think about.

``` haskell
class RemoteMemory m where
type Remote m e -- The remote information associated with an array

allocate :: m -> Int -> IO (Maybe (ArrayData e))
free :: m -> ArrayData e -> IO ()
attach :: m -> ArrayData e -> IO () -- Keep track of an existing host-side array in the Remote memory
push :: m -> ArrayData e -> IO ()
pull :: m -> ArrayData e -> IO ()
withRemote :: ArrayData e -> (Remote m e -> IO a) -> IO a

newtype LRU m = LRU m (... some structure used as an array table ...) (... some structure used as a nursery...)

instance RemoteMemory m => RemoteMemory (LRU m) where
type Remote (Lazy m) a = Remote m a
allocate (Lazy m n) sz =
if inNursery n sz then ... use array from nursery ... else
if ... not enough free space in device memory... then ... flush nursery ... else
... remove LRU array from table ...
free (Lazy m n) a = insertNursery n a
... the rest as expected ...

---------------------------------
-- In the CUDA backend.
newtype CudaMem = CudaMem

instance RemoteMemory CudaMem where
type Remote CudaMem a = DevicePtrs a
... All the functions trivially implememnted with just cudaMalloc, cudaFree and cudaMemCpy ...
```

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。