AxisCommunications / AxisCommunications/acap-native-sdk-examples

ARTPEC-9 mis-executes a residual Add fused into a following Conv2D

Open
#508 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C
Stars
83
Forks
45
Avg merge
3d 1h
Merged PRs (30d)
4

Description

An int8 TFLite graph that contains a residual ADD whose output feeds the next CONV_2D produces wrong results on the ARTPEC-9 DLPU. The same graph is correct on CPU and on ARTPEC-8. When the ADD result is materialised (declared as a model output, so the compiler cannot fuse it into the conv), the graph is correct on the ARTPEC-9 DLPU too.

Minimal repro:
Five ops, random weights: CONV_2D -> CONV_2D -> ADD (residual) -> CONV_2D.
- fused form: 99.9% of output values wrong on ARTPEC-9 (bit-exact on ARTPEC-8)
- ADD as an output: correct on ARTPEC-9

What does not fix it
Per-channel quantisation, symmetric ranges, padded 3x3 convs, removing MIN/MAX caps, layout reordering. Only materialising the ADD does.

Workaround in production
Every residual ADD is exported as an extra model output.

Seen on Q1726-LE, Q3558-LVE, Q6088, P3278, Q3556-LVE, AXIS OS 12.10 to 12.11.77, larod device a9-dlpu-tflite. Same files bit-exact on axis-a8-dlpu-tflite and cpu-tflite.

This finding was by debbuging the issue with CLAUDE, I let it create a minial repo.
(five stock layers, random weights, int8 per-tensor): input -> Conv2D 3x3 ReLU -> Conv2D 3x3 -> Add(input, .) -> ReLU -> Conv2D 1x1
[ethosn_repro.zip](https://github.com/user-attachments/files/31899314/ethosn_repro.zip)

Because i have a workaround I am happy, but I just want inform you guys, may there is something.

Helpdesk case: 02599166

Contributor guide

Open the contributing guide

Research direction

Start by running the minimal reproduction in ethosn_repro.zip and compare the fused and materialised-ADD graphs on ARTPEC-9, ARTPEC-8, and CPU using the device configurations named in the report. Done means explaining or correcting the ARTPEC-9 mismatch so the fused graph is bit-exact, or documenting the required limitation if the defect is outside this repository.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, tensorflow
Domain
embedded-iot, machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.