[Information] DirectML's bone-stupid method of reporting used memory deciphered, do what you will with it

TL;DR: Call torch_directml.get_gpu_memory() with tile size set to GPU memory in MB and the first element of the returned list is the percentage of memory in use. [0-1.0)

I've been messing around with directml trying to figure out why it's so amazingly messed up. All the user references to it online and my own quick test were just that it returned a big array of zeros. I think everybody sorta assumed like I did that "get_gpu_memory" would return the total memory somewhere in the array if it worked, or maybe tried it later and got the apparent garbage it outputs, and never messed with it during an actual program run.

I noticed that when I allocated a small tensor and called get_gpu_memory, the first array element went up slightly... that's a good sign. Then I created some more and it went up more. A little math later and I used that "tile size" parameter that defaults to 1MB, set it to 24567 to match the 24GB of video ram I have, and then created 5 1024x1024 fp32 tensors on the device, which should be ~4MB each depending on alignment and compute shader overheads.

First part is from memory, but the sizes are right and the other parts don't matter a whole lot:

```python
>>> import torch
>>> import torch_directml
>>>dev = torch.device(torch_directml.device()) _<-------- because I have no idea if this matters_
>>> torch.set_default_datatype(torch.float32)
>>> b = torch.randn(1024, 1024, device=dev)
>>> c = torch.ones(1024, 1024, device=dev)
>>> d = b * c
>>> e = torch.randn(1024, 1024, device=dev)
>>> _leaving out f because I can't remember what the syntax was or why it couldn't be called directly_
>>> g = e * d
>>> print(nat.get_gpu_memory(torch_directml.default_device(), 24576))
[0.0008533333311788738, 0.0, 0.0, 0.0, 0.0, 0.0]
>>> del b
>>> del c
>>> del d
>>> del e
>>> del f _<------ this was just a torch.nn function object, testing out GPU-based Log_Softmax_
>>> del g
>>> print(nat.get_gpu_memory(torch_directml.default_device(), 24576))
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
>>>
```

Yay, deleting everything freed it.

And whadda ya know, 0.00085333333 * 24576 = 20.97151991808, only 1MB more than I'd expect... but this is a GPU so who knows what alignment is required (OpenCL is 2k alignment) or how inefficient torch_directml is at compiling pytorch to shaders. I sent 3-4 matmuls over, and since I don't think DirectML really knows about the AMD matrix instructions yet (at least not on the version they're still using) it probably had to decompose that into a fairly large amount of code. I don't know what the "tile" reference really is.

If you do this with the default 1MB value, something else happens. I'm not sure what it represents but the array elements start filling up with 1.0 values from the start and fill up fast. Possibly it expands this array to meet / show each full use of the tile size given. I don't really understand why they'd express things that way since you can't, afaik, set the tile size nor does it matter when viewing available memory, and the length of the array is unrelated to anything I could figure out.

If a couple of people (with differing amounts of vram hopefully) could verify this isn't just the most bizarre fluke ever on my machine, getting the total memory is fairly easy if you're willing to shell out to an external utility (take your pick of clinfo, vulkaninfo, or hipinfo... even dxdiag /t if you're willing to wait around for 20s for it to spew a full system report into a text file. Most of these report available memory but since these are all low level APIs that track their own allocations and nobody else's (although I'm sure microsoft's page on video driver authoring explains how to check in detail and nobody read it as usual), every one of them thinks the GPU is more or less empty. CLinfo will give a max_allocation or similar though, vulkaninfo gives a "budget" for memoryHeaps[1] (and is harder to parse), and HIPinfo gives the most unrealistic number for available memory in memInfo.free (showing 99% on my machine). Not really optimal but there aren't any good modules to do this and clinfo / vulkaninfo should be installed on the machines of everybody who's still stuck using DirectML anyway.

or load a DLL (both of those sound unsafe, but torch loads kernel32.dll into torch.kernel32 on Windows and then leaves the namespace sitting around with the following mapped:
```python
>>> torch.kernel32.
torch.kernel32.AddDllDirectory(
torch.kernel32.LoadLibraryW(
torch.kernel32.LoadLibraryExW(
torch.kernel32.SetErrorMode(
```

Anyway I don't know if you feel like bothering with implementing this or not, or if my machine is just lucky, but it's a way to track DirectML memory use until MIOpen gets officially re-ported to Windows by the ROCM people (Radeon ProRender already includes MIOpen.dll so...)