tensorflow / tensorflow/tensorflow
tf.raw_ops.WriteScalarSummary can segfault on GPU instead of raising a Python exception
Open
@Venkat6871 is already working on this.
Since Mar 24, 2026.
2.21.0
awaiting PR merge
comp:ops
type:bug
- Dominant language
- C++
- Stars
- 200k
- Forks
- 76.9k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 433
Description
Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
2.21.0
Custom code
Yes
OS platform and distribution
Ubuntu 24.04.3 LTS x86_64
Mobile device
No response
Python version
3.10.20
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
13.0
GPU model and memory
No response
Current behavior?
System information
- TensorFlow version: 2.21.0
- Python version: 3.10.20
- OS: Linux 6.17.0-19-generic x86_64
- GPU: 4 x NVIDIA GeForce RTX 2080 Ti
- Reproduced in:
condaenvtf_test
Description
I found a case where tf.raw_ops.WriteScalarSummary crashes the Python process on GPU instead of raising a normal Python exception.
In batch validation this showed up as a generic signal_kill, but when I reran the same script standalone, it consistently reproduced as:
Segmentation fault (core dumped)
This looks like a hard crash rather than expected input validation behavior.
Reproducer
import tensorflow as tf
def test_write_scalar_summary():
logdir = "logs/"
writer = tf.summary.create_file_writer(logdir)
step = tf.constant(1, dtype=tf.int64)
tag = tf.constant("test_tag", dtype=tf.string)
values = [
tf.constant(0.0, dtype=tf.float32),
tf.constant(0.0, dtype=tf.float64),
tf.constant(0, dtype=tf.int32),
tf.constant(0, dtype=tf.int64),
tf.constant(0, dtype=tf.float16),
tf.constant([], dtype=tf.float32),
tf.constant([[0]], dtype=tf.float32),
tf.constant([[0, 0]], dtype=tf.float32),
]
with writer.as_default():
for value in values:
try:
tf.raw_ops.WriteScalarSummary(
writer=writer._resource,
step=step,
tag=tag,
value=value,
)
print(f"Successfully wrote summary for value: {value.numpy()}")
except Exception as e:
print(f"Failed to write summary for value: {value.numpy()} with error: {e}")
writer.close()
with tf.device('/GPU:0'):
test_write_scalar_summary()
### Standalone code to reproduce the issue
```shell
conda run --no-capture-output -n tf_test python reproduce.py
Actual behavior
The process terminates with a segfault:
Segmentation fault (core dumped)
Before the crash, TensorFlow initializes GPU devices normally.
Expected behavior
tf.raw_ops.WriteScalarSummary should reject unsupported inputs by raising a normal Python / TensorFlow exception such as InvalidArgumentError or ValueError, rather than crashing the whole process.
pip list
Package Version
------------------------ ----------
absl-py 2.4.0
aiohappyeyeballs 2.6.1
aiohttp 3.13.3
aiosignal 1.4.0
annotated-types 0.7.0
anyio 4.12.1
astunparse 1.6.3
async-timeout 5.0.1
attrs 25.4.0
cachetools 7.0.5
certifi 2026.2.25
charset-normalizer 3.4.5
colorlog 6.10.1
distro 1.9.0
exceptiongroup 1.3.1
flatbuffers 25.12.19
frozenlist 1.8.0
gast 0.7.0
google-pasta 0.2.0
grpcio 1.78.0
h11 0.16.0
h5py 3.14.0
httpcore 1.0.9
httpx 0.28.1
idna 3.11
jiter 0.13.0
keras 3.12.1
libclang 18.1.1
markdown-it-py 4.0.0
mdurl 0.1.2
ml_dtypes 0.5.4
multidict 6.7.1
namex 0.1.0
networkx 3.4.2
numpy 2.2.6
nvidia-cublas-cu12 12.9.1.4
nvidia-cuda-cupti-cu12 12.9.79
nvidia-cuda-nvcc-cu12 12.9.86
nvidia-cuda-nvrtc-cu12 12.9.86
nvidia-cuda-runtime-cu12 12.9.79
nvidia-cudnn-cu12 9.20.0.48
nvidia-cufft-cu12 11.4.1.4
nvidia-curand-cu12 10.3.10.19
nvidia-cusolver-cu12 11.7.5.82
nvidia-cusparse-cu12 12.5.10.65
nvidia-nccl-cu12 2.29.7
nvidia-nvjitlink-cu12 12.9.86
openai 0.28.0
opt_einsum 3.4.0
optree 0.19.0
packaging 26.0
pip 26.0.1
propcache 0.4.1
protobuf 7.34.0
pydantic 2.12.5
pydantic_core 2.41.5
Pygments 2.19.2
requests 2.32.5
rich 14.3.3
setuptools 82.0.1
six 1.17.0
sniffio 1.3.1
tensorflow 2.21.0
termcolor 3.3.0
tqdm 4.67.3
typing_extensions 4.15.0
typing-inspection 0.4.2
urllib3 2.6.3
wheel 0.46.3
wrapt 2.1.2
yarl 1.23.0
Relevant log output
(tf_test) user@user-Standard-PC-Q35-ICH9-2009:/home/lhj$ python callChainBuild/src_tensorflow/output_tensorflow/targeted_mutation/validation_sharded_full_20260322/potential_bugs/tf_raw_ops_WriteScalarSummary/0.py
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1774265664.422049 3732488 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
I0000 00:00:1774265664.492746 3732488 cpu_feature_guard.cc:227] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1774265665.815049 3732488 port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
I0000 00:00:1774265667.333059 3732488 gpu_device.cc:2043] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 9606 MB memory: -> device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5
I0000 00:00:1774265667.335750 3732488 gpu_device.cc:2043] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 9606 MB memory: -> device: 1, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:02:00.0, compute capability: 7.5
I0000 00:00:1774265667.338406 3732488 gpu_device.cc:2043] Created device /job:localhost/replica:0/task:0/device:GPU:2 with 9606 MB memory: -> device: 2, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:03:00.0, compute capability: 7.5
I0000 00:00:1774265667.341101 3732488 gpu_device.cc:2043] Created device /job:localhost/replica:0/task:0/device:GPU:3 with 9606 MB memory: -> device: 3, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:04:00.0, compute capability: 7.5
E0000 00:00:1774265667.602828 3732488 util.cc:131] oneDNN supports DT_HALF only on platforms with AVX-512. Falling back to the default Eigen-based implementation if present.
Successfully wrote summary for value: 0.0
Successfully wrote summary for value: 0.0
Successfully wrote summary for value: 0
Successfully wrote summary for value: 0
Successfully wrote summary for value: 0.0
Segmentation fault (core dumped)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.