dmlc / dmlc/decord

VideoLoader memory leak until program crash

Open
#218 1 comment 6 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
232
PR merge metrics
No merged PRs in 30d

Description

While iterating through the batches of the VideoLoader, the memory approaches 100% until eventually crashing. This occurs for all possible values of the shuffle parameter.

Code to Reproduce:

import numpy as np
from sklearn.preprocessing import OneHotEncoder
import decord as dc

ctx = dc.cpu(0)
batchsz = 10
shape = (batchsz, height, width, 3)

interval = 20
skip = 50

vl = dc.VideoLoader(videopaths, ctx=ctx, shape=shape, interval=interval, skip=skip, shuffle=2)

def frameGenerator():
while True:
vl.reset()
i = 1
for batch in vl:
result = i % 15
#k = [[video2label[p]] for p in np.array(batch[1][:,0])]
k = [[video2label[p]] for p in batch[1].asnumpy()[:,0]]
y_train = onehot.transform(k)
x_train = batch[0]
i += 1
yield x_train.asnumpy(), y_train

traingen = frameGenerator()
while True:
next(traingen)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.