NVIDIA / NVIDIA/Deep-Learning-Accelerator-SW
Accuracy of sigmoid layer's output drops a lot
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 236
- Forks
- 24
- PR merge metrics
- No merged PRs in 30d
Description
platform: Jetson AGX Orin 64GB
OS: 5.1.2
DLA: 3.12.1
Sigmoid layers are used as the output of the model and the input & output shape of sigmoid is (8, 3, 88, 160). I found the accuracy of fp16 dla model drops a lot when I use sigmoid as output layer. However, the outputs is consistent to torch outputs if the sigmoid is removed, with the cosine similarity close to 1.
I want to know what is the limitations on the use of sigmoid layers ?
Why does this loss of precision occur ?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported comparison on Jetson AGX Orin with the fp16 DLA model, using the stated sigmoid output shape of (8, 3, 88, 160) and comparing it with torch outputs. No repository file or test is named; done would be identifying the relevant sigmoid limitations and explaining the observed precision loss.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100