google / google/XNNPACK

XNNPACK delegate accepts rank-7 FULLY_CONNECTED then fails during Prepare

Open
#11,036 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
2.5k
Forks
560
Avg merge
1d 6h
Merged PRs (30d)
163

Description

## Summary

The default XNNPACK delegate claims a rank-7 `FULLY_CONNECTED` node and then
fails during delegate preparation. This aborts interpreter setup, although
the same FlatBuffer runs correctly with the builtin kernels when default
delegates are disabled.

## Environment

- LiteRT: `ai-edge-litert 2.1.6`
- TensorFlow converter: `tf-nightly 2.22.0.dev20260808` (also reproduced on
TensorFlow 2.20.0)
- Linux x86_64 CPU, Python 3.11

## Minimal reproduction

```python
import tensorflow as tf
from ai_edge_litert.interpreter import Interpreter, OpResolverType

x1 = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], shape=[1, 3, 2])

class Model(tf.keras.Model):
def __init__(self):
super().__init__()
self.w1 = tf.Variable([1.0] * 6)

def call(self, x):
w = tf.reshape(self.w1, (6, 1, 1, 1))
v = tf.reshape(x, (1, 1, 1, 6))
y = tf.tensordot(v, w, axes=1)
return tf.expand_dims(y, axis=-1) # output rank: 7

model = Model()
print("eager:", model(x1).shape, model(x1).numpy().flatten())
flatbuffer = tf.lite.TFLiteConverter.from_keras_model(model).convert()

def run(**kwargs):
interpreter = Interpreter(model_content=flatbuffer, **kwargs)
interpreter.allocate_tensors()
input_detail = interpreter.get_input_details()[0]
interpreter.set_tensor(input_detail["index"], x1.numpy())
interpreter.invoke()
return interpreter.get_tensor(interpreter.get_output_details()[0]["index"])

print(
"builtin only:",
run(
experimental_op_resolver_type=
OpResolverType.BUILTIN_WITHOUT_DEFAULT_DELEGATES
).flatten(),
)
print("default delegates:", run().flatten())
```

## Actual result

```text
eager: (1, 1, 1, 1, 1, 1, 1) [21.]
builtin only: [21.]
RuntimeError: failed to delegate FULLY_CONNECTED node #1
Node number 2 (TfLiteXNNPackDelegate) failed to prepare.
```

The default-delegate run raises from `allocate_tensors()` or `invoke()` rather
than returning a result.

## Expected result

The delegate should either support this node or decline to claim it during
partitioning/support checking. In the latter case, the builtin CPU kernel
should execute the node automatically, as demonstrated by the successful
`BUILTIN_WITHOUT_DEFAULT_DELEGATES` run. A preparation failure after the node
has been delegated makes an otherwise runnable model fail to load.

## Context

The same behavior was reproduced through both `tf.lite.Interpreter` and
`ai_edge_litert`. LiteRT maintainers reproduced the failure and asked that it
be tracked in XNNPACK: https://github.com/google-ai-edge/LiteRT/issues/9215

Contributor guide

Open the contributing guide

Research direction

Start by running the minimal Python reproduction with the default XNNPACK delegate and with BUILTIN_WITHOUT_DEFAULT_DELEGATES. Trace the XNNPACK FULLY_CONNECTED support-checking and preparation paths for the rank-7 node. Done means the default-delegate run either succeeds or declines the node so the builtin kernel returns [21.].

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.