microsoft / microsoft/onnxruntime-genai

"enable_profiling" in session_options is not working for dml

Open
#2,031 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ep:DML platform:mobile
Dominant language
C++
Stars
1.1k
Forks
354
Avg merge
2d 16h
Merged PRs (30d)
85

Description

Describe the bug
A clear and concise description of what the bug is.

"enable_profiling" in session_options is not working for dml, the data is missing after first round

To Reproduce
Steps to reproduce the behavior:

add "enable_profiling" in session_options

onnxruntime-genai-winml==0.12.2
onnxruntime-windowsml==1.23.2.202512041916

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Sample data

{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :10830,"ts" :9,"ph" : "X","name" :"model_loading_uri","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :1431772,"ts" :13040,"ph" : "X","name" :"session_initialization","args" : {}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :69,"ts" :2579165,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Shape_kernel_time","args" : {"parameter_size" : "0","provider" : "DmlExecutionProvider","op_name" : "Shape","input_type_shape" : [{"int64":[1,200]}],"node_index" : "3","output_type_shape" : [{"int64":[2]}],"activation_size" : "1600","output_size" : "16","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :17,"ts" :2579247,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather_kernel_time","args" : {"parameter_size" : "8","provider" : "CPUExecutionProvider","op_name" : "Gather","input_type_shape" : [{"int64":[2]},{"int64":[]}],"node_index" : "4","output_type_shape" : [{"int64":[]}],"activation_size" : "16","output_size" : "8","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :16,"ts" :2579271,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather/Cast_kernel_time","args" : {"parameter_size" : "0","provider" : "CPUExecutionProvider","op_name" : "Cast","input_type_shape" : [{"int64":[]}],"node_index" : "5","output_type_shape" : [{"int32":[]}],"activation_size" : "8","output_size" : "4","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :479132,"ts" :2579294,"ph" : "X","name" :"DmlFusedNode_0_0_kernel_time","args" : {"parameter_size" : "1237356552","provider" : "DmlExecutionProvider","op_name" : "DmlFusedNode_0_0","input_type_shape" : [{"int32":[]},{"int64":[1]},{"int64":[1,200]},{"float16":[131072,32]},{"int64":[1,13]},{"uint8":[128256,64,16]},{"float16":[8208384]},{"float16":[128256,2048]},{"uint8":[512,64,16]},{"float16":[32768]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[512,64,16]},{"float16":[32768]},{"float16":[2048]},{"uint8":[2048,256,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"float16":[2048]},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :479290,"ts" :2579160,"ph" : "X","name" :"SequentialExecutor::Execute","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :479415,"ts" :2579068,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :57,"ts" :4035530,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Shape_kernel_time","args" : {"parameter_size" : "0","provider" : "DmlExecutionProvider","op_name" : "Shape","input_type_shape" : [{"int64":[1,200]}],"node_index" : "3","output_type_shape" : [{"int64":[2]}],"activation_size" : "1600","output_size" : "16","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :23,"ts" :4035600,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather_kernel_time","args" : {"parameter_size" : "8","provider" : "CPUExecutionProvider","op_name" : "Gather","input_type_shape" : [{"int64":[2]},{"int64":[]}],"node_index" : "4","output_type_shape" : [{"int64":[]}],"activation_size" : "16","output_size" : "8","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :20,"ts" :4035637,"ph" : "X","name" :"/model/attn_mask_reformat/attn_mask_subgraph/Gather/Cast_kernel_time","args" : {"parameter_size" : "0","provider" : "CPUExecutionProvider","op_name" : "Cast","input_type_shape" : [{"int64":[]}],"node_index" : "5","output_type_shape" : [{"int32":[]}],"activation_size" : "8","output_size" : "4","thread_scheduling_stats" : {"main_thread": {"thread_pool_name": "session-5-intra-op", "thread_id": "152600", "block_size": [], "core": 2, "Distribution": 0, "DistributionEnqueue": 0, "Run": 0, "Wait": 0, "WaitRevoke": 0}, "sub_threads": {"162504": {"num_run": 0, "core": -1},"150268": {"num_run": 0, "core": -1},"161924": {"num_run": 0, "core": -1}}}}},
{"cat" : "Node","pid" :148092,"tid" :152600,"dur" :294501,"ts" :4035665,"ph" : "X","name" :"DmlFusedNode_0_0_kernel_time","args" : {"parameter_size" : "1237356552","provider" : "DmlExecutionProvider","op_name" : "DmlFusedNode_0_0","input_type_shape" : [{"int32":[]},{"int64":[1]},{"int64":[1,200]},{"float16":[131072,32]},{"int64":[1,1]},{"uint8":[128256,64,16]},{"float16":[8208384]},{"float16":[128256,2048]},{"uint8":[512,64,16]},{"float16":[32768]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[2048,64,16]},{"float16":[131072]},{"uint8":[512,64,16]},{"float16":[32768]},{"float16":[2048]},{"uint8":[2048,256,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"uint8":[8192,64,16]},{"float16":[524288]},{"float16":[2048]},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :294663,"ts" :4035525,"ph" : "X","name" :"SequentialExecutor::Execute","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :294816,"ts" :4035399,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :66,"ts" :4362725,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :65,"ts" :4394836,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :62,"ts" :4427730,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :67,"ts" :4461076,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :66,"ts" :4493765,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :63,"ts" :4525739,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :64,"ts" :4558472,"ph" : "X","name" :"model_run","args" : {}},
{"cat" : "Session","pid" :148092,"tid" :152600,"dur" :86,"ts" :4590509,"ph" : "X","name" :"model_run","args" : {}},

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Smartphone (please complete the following information):

  • Device: [e.g. iPhone6]
  • OS: [e.g. iOS8.1]
  • Browser [e.g. stock browser, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with enable_profiling in session_options using onnxruntime-genai-winml 0.12.2 and onnxruntime-windowsml 1.23.2.202512041916. Compare the profiling data from the first round with later model runs; done means profiling data is retained after the first round for the DmlExecutionProvider.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.