cuda memory leak
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 249
- Forks
- 37
- PR merge metrics
- No merged PRs in 30d
Description
**when this operator is called repeatedly during iterations, the GPU memory usage keeps increasing, finally the error "cuda 3D Allocation error:out of memory "occurs. And this phenomenon only occurs in ubuntu system, not in window system **
class projector_t(nn.Module):
def __init__(self):
super(projector_t, self).__init__()
def forward(self, sino, index, batchSize, d_angle=0):
device_name = "cuda:1"
device = torch.device(device_name)
proj = Projector(forward_project=False, use_static=True, use_gpu=True, gpu_device=device, batch_size=batchSize)
# proj = FBP(forward_FBP=True, use_static=True, use_gpu=True, gpu_device=device, batch_size=batchSize)
numCols = 1024
numTurns = 1024 / 1024
numAngles = int(1024 * numTurns)
pixelHeight = 1.12
pixelWidth = 1.12
# Set the number of detector rows
numRows = 32
phis = proj.leapct.setAngleArray(numAngles, 360.0 * numTurns)
# Set the scanner geometry
proj.leapct.set_conebeam(numAngles, numRows, numCols, pixelHeight, pixelWidth, 0.5 * (numRows - 1),
0.5 * (numCols - 1) - 4.4912, phis+d_angle, 570, 1005, tau=0.0)
proj.leapct.set_curvedDetector()
# Set the helical pitch.
proj.leapct.set_normalizedHelicalPitch(1.6)
pixels = 512
slices = 5
voxelWidth = 0.6
voxelHeight = 1.0
proj.leapct.set_volume(pixels, pixels, slices, voxelWidth, voxelHeight)
proj.allocate_batch_data()
g = sino
f = proj.leapct.allocateVolume()
f = torch.from_numpy(f).to(device)
# proj.leapct.set_FORBILD(f, True)
f[:] = 0.0
# Reconstruct the data
proj.leapct.FBP(g, f)
# f = proj(g)
del sino, g, proj
return f
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the shown projector_t.forward path and its repeated Projector construction, then measure GPU memory across repeated calls on Ubuntu and Windows. Done means repeated iterations no longer steadily increase GPU usage or end in a CUDA 3D allocation out-of-memory error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100