tensorflow / tensorflow/tflite-micro

Standardize TFLM Reference Kernels to Single Rounding Requantization

Open
#3,252 1 comment 0 reactions 1 assignee View on GitHub

@veblush is already working on this.

Since Feb 10, 2026.

Dominant language
C++
Stars
3.1k
Forks
1.1k
Avg merge
1d 5h
Merged PRs (30d)
43

Description

Issue

TFLM's Reference Kernels (and TFLite's one) use a legacy "Double Rounding" algorithm, resulting in an approximately ~1.8% off-by-one error rate. This diverges from the mathematically correct "Single Rounding" used by TFLite's Optimized kernels, primarily XNNPack and ruy, which serve as the "Golden Reference." Context: Single-round was introduced in TFLite by https://github.com/tensorflow/tensorflow/pull/50290 to address https://github.com/tensorflow/tensorflow/issues/25087 in 2021 but it wan't enabled by default due to concerns about regression.

This inconsistency creates a critical "Validation Trap":

  • Validation: Developers validate models using TFLite in Python (correct Single Rounding).
  • Deployment: Deploying to TFLM (incorrect Double Rounding) causes output mismatches, leading to unexpected accuracy drops and engineering churn.

Proposal: Standardize Reference Kernels to Single Rounding

I propose standardizing the Reference Kernel behavior to Single Rounding, ensuring TFLM matches the TFLite behavior.

Action Plan:

  1. Enabling TFLITE_SINGLE_ROUNDING by default in Bazel & Makefile build. (Optionally, we can consider introducing a new switch to use the double-rounding)
  2. Vendor Alignment and Coordination: Propagate this standard to optimized TFLM vendor kernels (e.g., ARM CMSIS-NN, Cadence Xtensa). Coordination is essential to ensure these paths align with the new mathematical standard, maintaining ecosystem consistency.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.