decreasing the amount of ram used on gpu

Hello,

I'm trying to decrease the amount of ram used on the gpu for inference.

i've used the supplied cfgs.py that uses resnet101,
it uses up about 6G during training,
then i've frozen the graph -> created a 190MB frozen file,
i then use this frozen file from CPP , it consumes about 800MB on gpu for inference.

i then tried using the cfgs_res50.py, this lead to 3.5G of ram using on gpu during training,
a 115MB frozen graph file, and ram consumed on gpu during inference (from cpp) has not decreased significantly ( about 720M).

can you suggest a method to decrease the ram consumption on gpu ?
( maybe some other property of the network ? )

thanks,
Omer.

Start by comparing cfgs.py and cfgs_res50.py, then inspect how the frozen graph is loaded and run from C++. Measure GPU RAM during training and inference for both graphs, using the reported 800 MB and 720 MB figures as a baseline. Done would require a demonstrated way to reduce inference GPU usage, but the issue does not define a target or specific implementation area.

Written by the indexing model from the issue text.