Sharing memory pools in CUDAGraphs has non-trivial constraints for memory reuse

@ali-alshaar7 is already working on this.

Since Feb 25, 2025.

*Note*: If you have a model or program that is not supported yet but should be, please use the program coverage template.

## 🐛 Bug

In theory, [sharing memory pools across CUDAGraphs](https://github.com/Lightning-AI/lightning-thunder/pull/1790) should decrease allocated memory, but in our experiments, it doesn't.

### To Reproduce

follow the test in the above PR.

### Expected behavior

The following minimal repro using pure torch cg does show difference between setting men_pool = None and men_pool = g1.pool(), but the difference is very small.

```
# with pool sharing: 15585.885696
# without pool sharing: 15586.934272
```

```py
import torch
import litgpt
from torch.testing import make_tensor
from functools import partial

device = torch.device("cuda")

cfg = litgpt.Config.from_name("open_llama_3b", n_layer=10)
with device:
make = partial(make_tensor, low=0, high=255, device=device, dtype=torch.long, requires_grad=False)
shape = (1,) + (cfg.block_size,)
shape2 = (1,) + (cfg.block_size//2,)

x = make(shape)
y = make(shape2)
m = litgpt.GPT(cfg)

workload = m

g1 = torch.cuda.CUDAGraph()
g2 = torch.cuda.CUDAGraph()

for _ in range(3):
with torch.no_grad():
static_out_1 = workload(x)
static_out_2 = workload(y)

with torch.cuda.graph(g1):
static_out_1.copy_(workload(x))

mem_pool = None # or g1.pool() for pool sharing

with torch.cuda.graph(g2, pool=mem_pool):
static_out_2.copy_(workload(y))

# with pool sharing: 15585.885696
# without pool sharing: 15586.934272
print(torch.cuda.max_memory_allocated() / 1e6)

```

### Environment

- PyTorch Version (e.g., 1.0):
- OS (e.g., Linux):
- How you installed PyTorch (`conda`, `pip`, source):
- Build command you used (if compiling from source):
- Python version:
- CUDA/cuDNN version:
- GPU models and configuration: T4 GPU on Ligtning Studio
- Any other relevant information:

### Additional context