microsoft / microsoft/onnxruntime

I want to get the“hidden_states” data in the Bert model

Open
#20,664 3 comments 0 reactions 0 assignees View on GitHub
feature request model:transformer
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the feature request

I'm running a TTS project where the Bert model needs to get the“hidden_states” component of the middle layer output for later processing. I'm using C + + and can't seem to find a way to get the middle layer output, any suggestions?
![1715582733737](https://github.com/microsoft/onnxruntime/assets/74243065/daeff032-1af6-4c71-87af-083a0d47eb28)

### Describe scenario use case

This method is handy for using the Bert Model, because hidden_states are also useful features, which can be implemented in the Python version of Transformer, but is difficult to implement in C + + .

Contributor guide

Open the contributing guide

Research direction

Start by reviewing ONNX Runtime's C++ inference APIs and the BERT model path to determine how intermediate hidden-state outputs are represented. Compare this with the Python Transformer behavior described in the issue. Done means the supported approach for retrieving middle-layer hidden states in C++ is clearly defined and covered by appropriate validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.