Fast inference with low RAM usage

请提出你的问题 Please ask your question

We have a project where we want to use PaddlePadle or Paddle-Lite
We want to use it for OCR, background matting and face detection.
Our platform is a Nvidia Jetson TX2 NX with 4 GB RAM.

Our requirements are following:

• Fast inference on GPU
• Low RAM usage (maximum 1 GB)
• CUDA 10.2
• CUDNN 8.0
• TENSORRT 7.1.3

First, we tried PaddlePaddle:
PaddleOCR with PaddlePaddle takes 700ms for our test document, that’s OK.
But it uses a lot of Ram (over 2 GB before loading the models).
That’s too much, our Software needs more free RAM!

Then we tried Paddle-Lite:
PaddleOCR with Paddle-Lite takes 5500ms for our test document, that’s too slow.
But it uses less RAM, that’s good!

What we need is either Paddle-Lite using the GPU so it is faster or PaddlePaddle using less RAM.

Can you assist us in this problem?

1. Can you give us a compiled version of PaddlePaddle or Paddle-Lite that meets our requirements?
2. Can you tell us the changes we need to make in the sources to compile one of these versions by our self?