microsoft / microsoft/onnxruntime-genai
"enable_profiling" in session_options is not working for dml
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.1k
- Forks
- 354
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 85
Description
Describe the bug
A clear and concise description of what the bug is.
"enable_profiling" in session_options is not working for dml, the data is missing after first round
To Reproduce
Steps to reproduce the behavior:
add "enable_profiling" in session_options
onnxruntime-genai-winml==0.12.2
onnxruntime-windowsml==1.23.2.202512041916
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Sample data
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :10830,"ts" :9,"ph" : "X","name" :"model_loading_uri","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :1431772,"ts" :13040,"ph" : "X","name" :"session_initialization","args" : {}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :69,"ts" :2579165,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Shape_kernel_time","args" : {"parameter_size" : "0","provider" : "DmlExecutionProvider","op_name" : "Shape","input_type_shape" : [{"int64":[1,200]}],"node_index" : "3","output_type_shape" : [{"int64":[2]}],"activation_size" : "1600","output_size" : "16","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :17,"ts" :2579247,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather_kernel_time","args" : {"parameter_size" : "8","provider" : "CPUExecutionProvider","op_name" : "Gather","input_type_shape" : [{"int64":[2]},{"int64":[]}],"node_index" : "4","output_type_shape" : [{"int64":[]}],"activation_size" : "16","output_size" : "8","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :16,"ts" :2579271,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather/Cast_kernel_time","args" : {"parameter_size" : "0","provider" : "CPUExecutionProvider","op_name" : "Cast","input_type_shape" : [{"int64":[]}],"node_index" : "5","output_type_shape" : [{"int32":[]}],"activation_size" : "8","output_size" : "4","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :479132,"ts" :2579294,"ph" : "X","name" :"DmlFusedNode_0_0_kernel_time","args" : {"parameter_size" : "1237356552","provider" : "DmlExecutionProvider","op_name" : "DmlFusedNode_0_0","input_type_shape" : [{"int32":[]},{"int64":[1]},{"int64":[1,200]},{"float16":[131072,32]},{"int64":[1,13]},{"uint8":[128256,64,16]},{"float16":[8208384]},{"float16":[128256,2048]},{"uint8":[512,64,16]},{"float16":[32768]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[512,64,16]},{"float16":[32768]},{"float16":[2048]},{"uint8":[2048,256,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"float16":[2048]},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :479290,"ts" :2579160,"ph" : "X","name" :"SequentialExecutor::Execute","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :479415,"ts" :2579068,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :57,"ts" :4035530,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Shape_kernel_time","args" : {"parameter_size" : "0","provider" : "DmlExecutionProvider","op_name" : "Shape","input_type_shape" : [{"int64":[1,200]}],"node_index" : "3","output_type_shape" : [{"int64":[2]}],"activation_size" : "1600","output_size" : "16","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :23,"ts" :4035600,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather_kernel_time","args" : {"parameter_size" : "8","provider" : "CPUExecutionProvider","op_name" : "Gather","input_type_shape" : [{"int64":[2]},{"int64":[]}],"node_index" : "4","output_type_shape" : [{"int64":[]}],"activation_size" : "16","output_size" : "8","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :20,"ts" :4035637,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather/Cast_kernel_time","args" : {"parameter_size" : "0","provider" : "CPUExecutionProvider","op_name" : "Cast","input_type_shape" : [{"int64":[]}],"node_index" : "5","output_type_shape" : [{"int32":[]}],"activation_size" : "8","output_size" : "4","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :294501,"ts" :4035665,"ph" : "X","name" :"DmlFusedNode_0_0_kernel_time","args" : {"parameter_size" : "1237356552","provider" : "DmlExecutionProvider","op_name" : "DmlFusedNode_0_0","input_type_shape" : [{"int32":[]},{"int64":[1]},{"int64":[1,200]},{"float16":[131072,32]},{"int64":[1,1]},{"uint8":[128256,64,16]},{"float16":[8208384]},{"float16":[128256,2048]},{"uint8":[512,64,16]},{"float16":[32768]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[512,64,16]},{"float16":[32768]},{"float16":[2048]},{"uint8":[2048,256,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"float16":[2048]},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :294663,"ts" :4035525,"ph" : "X","name" :"SequentialExecutor::Execute","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :294816,"ts" :4035399,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :66,"ts" :4362725,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :65,"ts" :4394836,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :62,"ts" :4427730,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :67,"ts" :4461076,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :66,"ts" :4493765,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :63,"ts" :4525739,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :64,"ts" :4558472,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :86,"ts" :4590509,"ph" : "X","name" :"model_run","args" : {}},
Desktop (please complete the following information):
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
Smartphone (please complete the following information):
- Device: [e.g. iPhone6]
- OS: [e.g. iOS8.1]
- Browser [e.g. stock browser, safari]
- Version [e.g. 22]
Additional context
Add any other context about the problem here.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with enable_profiling in session_options using onnxruntime-genai-winml 0.12.2 and onnxruntime-windowsml 1.23.2.202512041916. Compare the profiling data from the first round with later model runs; done means profiling data is retained after the first round for the DmlExecutionProvider.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100