tensorflow / tensorflow/tensorflow

tf.nn.separable_conv2d causes process abort (SIGABRT) in MklLayoutRewritePass when strides contains INT64_MAX

Open
#124,952 3 comments 0 reactions 1 assignee View on GitHub

@Kayyuri is already working on this.

Since Sep 10, 2026.

2.21.0 comp:core stat:awaiting response type:bug
Dominant language
C++
Stars
200k
Forks
76.9k
Avg merge
2d 3h
Merged PRs (30d)
433

Description

Issue type

Bug

Have you reproduced the bug with TensorFlow Nightly?

Yes

Source

source

TensorFlow version

2.21.0

Custom code

Yes

OS platform and distribution

Windows 10

Mobile device

No response

Python version

No response

Bazel version

No response

GCC/compiler version

No response

CUDA/cuDNN version

No response

GPU model and memory

No response

Current behavior?

Passing a strides list with an out-of-range integer value (such as 9223372036854775807 / INT64_MAX) to tf.nn.separable_conv2d causes a hard assertion failure inside the oneDNN/MKL layout graph optimization pass (MklLayoutRewritePass::CopyAttrsConv).

During graph node attribute parsing, GetNodeAttr fails because the provided value exceeds the valid int32 bounds, triggering a TF_CHECK_OK macro abort:
Check failed: ... is OK (INVALID_ARGUMENT: Attr strides has value 9223372036854775807 out of range for an int32)

Instead of aborting the process via SIGABRT, TensorFlow should perform input validation on stride values at the op entry level and raise a catchable tf.errors.InvalidArgumentError.

poc.py

Standalone code to reproduce the issue
import tensorflow as tf

input_tensor = tf.random.normal([2, 32, 32, 3])
depthwise_filter = tf.constant([-392347.53], shape=(1,))
pointwise_filter = tf.random.normal([1, 1, 6, 4])

# Extreme stride value triggering int32 overflow in oneDNN layout pass
strides = [1, 9223372036854775807, -8, -1]
padding = 'VALID'
data_format = 'NHWC'
dilations = [1, 1]

# Triggers fatal assertion failure in mkl_layout_pass.cc
output = tf.nn.separable_conv2d(
    input_tensor, 
    depthwise_filter, 
    pointwise_filter, 
    strides, 
    padding, 
    data_format=data_format, 
    dilations=dilations
)

Observed Behavior
The Python interpreter process aborts with a fatal check failure trace:
F0000 00:00:1786280609.175683 571 mkl_layout_pass.cc:2703] Check failed: (::tsl::TfCheckOkDeprecationMarker(), GetNodeAttr(orig_node->def(), "strides", &strides)) is OK (INVALID_ARGUMENT: Attr strides has value 9223372036854775807 out of range for an int32)
*** Check failure stack trace: ***
    @ 0x7fb11e25b314 absl::lts_20250814::log_internal::LogMessage::SendToLog()
    @ 0x7fb11bcfc3b2 tensorflow::MklLayoutRewritePass::CopyAttrsConv()
    @ 0x7fb11bcfbf86 tensorflow::MklLayoutRewritePass::CopyAttrsConvCheckConstFilter()
    @ 0x7fb11bd06253 tensorflow::MklLayoutRewritePass::RewriteNodeForJustOpNameChange()
    @ 0x7fb11bd06986 tensorflow::MklLayoutRewritePass::RewriteNode()
    @ 0x7fb11bd0979b tensorflow::MklLayoutRewritePass::RunPass()
Aborted (core dumped)

Expected Behavior
TensorFlow should validate that stride values fall within the acceptable int32 range before running optimization passes, raising a catchable exception:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Attr strides has value 9223372036854775807 out of range for an int32
Relevant log output
hacr@DESKTOP-NSBKSBB:/mnt/d/Speech$ python3 poc.py
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1786280603.970014     571 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
I0000 00:00:1786280603.972274     571 cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
I0000 00:00:1786280604.101628     571 cpu_feature_guard.cc:227] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1786280608.176580     571 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
I0000 00:00:1786280608.182197     571 cudart_stub.cc:31] Could not find cuda drivers on your machine, GPU will not be used.
E0000 00:00:1786280609.037613     571 cuda_platform.cc:52] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: UNKNOWN ERROR (303)
F0000 00:00:1786280609.175683     571 mkl_layout_pass.cc:2703] Check failed: (::tsl::TfCheckOkDeprecationMarker(), GetNodeAttr(orig_node->def(), "strides", &strides)) is OK (INVALID_ARGUMENT: Attr strides has value 9223372036854775807 out of range for an int32)
[symbolize_elf.inc : 379] RAW: Unable to get high fd: rc=0, limit=1024
*** Check failure stack trace: ***
    @     0x7fb11e25b314  absl::lts_20250814::log_internal::LogMessage::SendToLog()
    @     0x7fb11e25b296  absl::lts_20250814::log_internal::LogMessage::Flush()
    @     0x7fb11bcfc3b2  tensorflow::MklLayoutRewritePass::CopyAttrsConv()
    @     0x7fb11bcfbf86  tensorflow::MklLayoutRewritePass::CopyAttrsConvCheckConstFilter()
    @     0x7fb11bd06253  tensorflow::MklLayoutRewritePass::RewriteNodeForJustOpNameChange()
    @     0x7fb11bd06986  tensorflow::MklLayoutRewritePass::RewriteNode()
    @     0x7fb11bd0979b  tensorflow::MklLayoutRewritePass::RunPass()
    @     0x7fb11bd0a3bc  tensorflow::MklLayoutRewritePass::Run()
    @     0x7fb11bda7bee  tensorflow::OptimizationPassRegistry::RunGrouping()
    @     0x7fb11bd46b58  tensorflow::PreprocessAndPartitionGraph()
    @     0x7fb11bd21630  tensorflow::ProcessFunctionLibraryRuntime::InstantiateMultiDevice()
    @     0x7fb11bd2812b  tensorflow::ProcessFunctionLibraryRuntime::Instantiate()
    @     0x7fb114a6dcff  tensorflow::KernelAndDeviceFunc::InstantiateFunc()
    @     0x7fb114a6df82  tensorflow::KernelAndDeviceFunc::Init()
    @     0x7fb114a2408c  tensorflow::(anonymous namespace)::GetOrCreateKernelAndDevice()
    @     0x7fb114a1cf90  tensorflow::(anonymous namespace)::EagerLocalExecute()
    @     0x7fb114a1b1d0  tensorflow::DoEagerExecute()
    @     0x7fb114a1eb2b  tensorflow::EagerExecute()
    @     0x7fb11454c247  tensorflow::EagerOperation::Execute()
    @     0x7fb114a6bbb7  tensorflow::CustomDeviceOpHandler::Execute()
    @     0x7fb111b7e4e5  TFE_Execute
    @     0x7fb0e1a2a8f1  TFE_Py_FastPathExecute_C()
    @     0x7fb0de6da493  pybind11::detail::argument_loader<>::call<>()
    @     0x7fb0de6da3cf  pybind11::cpp_function::initialize<>()::{lambda()#1}::__invoke()
    @     0x7fb0de6afa79  pybind11::cpp_function::dispatcher()
    @           0x5821ef  (unknown)
    @           0x5492b5  _PyObject_MakeTpCall
    @           0x5d7389  _PyEval_EvalFrameDefault
    @           0x54ab02  _PyObject_Call_Prepend
    @           0x5a37f8  (unknown)
    @           0x5492b5  _PyObject_MakeTpCall
    @           0x5d7389  _PyEval_EvalFrameDefault
    @           0x5d58bb  PyEval_EvalCode
    @           0x608b52  (unknown)
    @           0x6b5263  (unknown)
    @           0x6b4fca  _PyRun_SimpleFileObject
    @           0x6b4dff  _PyRun_AnyFileObject
    @           0x6bce65  Py_RunMain
    @           0x6bc94d  Py_BytesMain
    @     0x7fb11f15a1ca  (unknown)
    @     0x7fb11f15a28b  __libc_start_main
    @           0x657b05  _start
Aborted (core dumped)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.