dmlc / dmlc/decord

Question about the efficiency with different num_threads set

Open
#332 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.5k
Forks
232
PR merge metrics
No merged PRs in 30d

Description

I am trying to push for higher frame extraction speed for a UHD video with 30M bitrate as the 12 frames are extracted by

`frame_batch = vr.get_batch(frame_indices).asnumpy()`

I am using a Xeon Platinum 8360Y, which has 36 cores, and using `cProfile` to track the execution time of each function call, and I surprisingly find that when I set

* `num_thread=0` (which is auto), the line of code takes 22.733s
* `num_thread=1`, 12.784s
* `num_thread=2`, 14.255s
* `num_thread=4`, 15.855s
* `num_thread=6`, 17.310s
* `num_thread=8`, 18.587s
* `num_thread=10`, 19.813s
* `num_thread=12`, 20.735s
* `num_thread=16`, 22.702s
* `num_thread=24`, 25.357s
* `num_thread=36`, 25.763s
* `num_thread=48`, 25.817s
* `num_thread=64`, 26.388s
* `num_thread=72`, 26.486s

Does this mean that the less threads, the better running speed?

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the reported timings for vr.get_batch(frame_indices).asnumpy() across the listed num_thread values with the UHD, 30Mbit video and cProfile. Start by tracing how num_thread is handled during get_batch; done means explaining the scaling behavior and identifying whether an actionable performance issue exists.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.