XNNPACK delegate accepts rank-7 FULLY_CONNECTED then fails during Prepare
- Dominant language
- C
- Stars
- 2.5k
- Forks
- 560
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 163
Description
## Summary
The default XNNPACK delegate claims a rank-7 `FULLY_CONNECTED` node and then
fails during delegate preparation. This aborts interpreter setup, although
the same FlatBuffer runs correctly with the builtin kernels when default
delegates are disabled.
## Environment
- LiteRT: `ai-edge-litert 2.1.6`
- TensorFlow converter: `tf-nightly 2.22.0.dev20260808` (also reproduced on
TensorFlow 2.20.0)
- Linux x86_64 CPU, Python 3.11
## Minimal reproduction
```python
import tensorflow as tf
from ai_edge_litert.interpreter import Interpreter, OpResolverType
x1 = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], shape=[1, 3, 2])
class Model(tf.keras.Model):
def __init__(self):
super().__init__()
self.w1 = tf.Variable([1.0] * 6)
def call(self, x):
w = tf.reshape(self.w1, (6, 1, 1, 1))
v = tf.reshape(x, (1, 1, 1, 6))
y = tf.tensordot(v, w, axes=1)
return tf.expand_dims(y, axis=-1) # output rank: 7
model = Model()
print("eager:", model(x1).shape, model(x1).numpy().flatten())
flatbuffer = tf.lite.TFLiteConverter.from_keras_model(model).convert()
def run(**kwargs):
interpreter = Interpreter(model_content=flatbuffer, **kwargs)
interpreter.allocate_tensors()
input_detail = interpreter.get_input_details()[0]
interpreter.set_tensor(input_detail["index"], x1.numpy())
interpreter.invoke()
return interpreter.get_tensor(interpreter.get_output_details()[0]["index"])
print(
"builtin only:",
run(
experimental_op_resolver_type=
OpResolverType.BUILTIN_WITHOUT_DEFAULT_DELEGATES
).flatten(),
)
print("default delegates:", run().flatten())
```
## Actual result
```text
eager: (1, 1, 1, 1, 1, 1, 1) [21.]
builtin only: [21.]
RuntimeError: failed to delegate FULLY_CONNECTED node #1
Node number 2 (TfLiteXNNPackDelegate) failed to prepare.
```
The default-delegate run raises from `allocate_tensors()` or `invoke()` rather
than returning a result.
## Expected result
The delegate should either support this node or decline to claim it during
partitioning/support checking. In the latter case, the builtin CPU kernel
should execute the node automatically, as demonstrated by the successful
`BUILTIN_WITHOUT_DEFAULT_DELEGATES` run. A preparation failure after the node
has been delegated makes an otherwise runnable model fail to load.
## Context
The same behavior was reproduced through both `tf.lite.Interpreter` and
`ai_edge_litert`. LiteRT maintainers reproduced the failure and asked that it
be tracked in XNNPACK: https://github.com/google-ai-edge/LiteRT/issues/9215
Contributor guide
Research direction
Start by running the minimal Python reproduction with the default XNNPACK delegate and with BUILTIN_WITHOUT_DEFAULT_DELEGATES. Trace the XNNPACK FULLY_CONNECTED support-checking and preparation paths for the rank-7 node. Done means the default-delegate run either succeeds or declines the node so the builtin kernel returns [21.].
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100