complete kernel hung on starting GUI app: kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* V3D_ERR_STAT: 0x00001000

Reproduce on a Raspberry Pi 4 running Raspberry Pi OS Bookworm under Wayland using the attached history.zip and octave --gui. Start by examining the v3d DRM hang messages in journalctl and compare behavior under Wayland and X11. Done means launching the GUI with the reproducing history file no longer hangs the kernel or requires a hard reboot.

Describe the bug

I recently encountered an unexpected crash of the Octave GUI, which I documented in this issue. After some investigation, I found that the problem was caused by the contents of the file ~/.local/share/octave/history which should be displayed on command history sub-window. In that case, simply deleting the file resolved the issue. However, I wanted to understand what specifically in the file caused the crash, so I began inspecting its contents.

It turns out the problem was triggered by a very long line - approximately 28'000 characters, representing an array initialization. I then tried reducing the line length to determine the threshold at which the issue occurs. I found that Octave GUI crashes somewhere between 4'000 and 5'000 characters.

More critically, while testing within that range, I encountered an even more severe issue: placing a history file with a line of a certain length causes a complete system hang, requiring a hard reboot.

The kernel completely hang, GUI stops to respond and the system didn't respond even on network connections. After reboot you can find 4 log records with error before kernel hang:

journalctl log:

May 27 11:59:10 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* Resetting GPU for hang.
May 27 11:59:10 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* V3D_ERR_STAT: 0x00001000
May 27 11:59:11 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* Resetting GPU for hang.
May 27 11:59:11 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* V3D_ERR_STAT: 0x00001000

I can stable reproduce this issue simply by placing the attached history file to ~/.local/share/octave/ folder and launching Octave GUI.

I suspect there is some issue with V3D video driver and possibly Qt5 libraries, probably some kind of security issue with buffer overflow.

The issue has been reproduced on Raspberry Pi OS Bookworm aarch64 (with all latest APT updates), running under the Wayland display server with the Labwc compositor. Based on testing for mentioned issue with octave crash, the same behavior is also observed when using the Wayfire compositor under Wayland. When running under X11, the behavior differs slightly but still results in problematic behavior.

Steps to reproduce the issyue

1. Build and install octave

2. unzip history file from attached ZIP archive to the folder ~/.local/share/octave/:

history.zip

3. Launch Octave GUI (from start menu, or just with terminal octave --gui)

Expected result: The Linux kernel continue running with no issue

Actual result: The Linux kernel completely hang and stops to work.

Device (s)

Raspberry Pi 4 Mod. B

System

$ cat /etc/rpi-issue
Raspberry Pi reference 2023-09-22
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 40f37458ae7cadea1aec913ae10b5e7008ebce0a, stage4

$ vcgencmd version
Apr 30 2025 13:33:39
Copyright (c) 2012 Broadcom
version 5560078dcc8591a00f57b9068d13e5544aeef3aa (clean) (release) (start)

$ uname -a
Linux rzx 6.12.25+rpt-rpi-v8 #1 SMP PREEMPT Debian 1:6.12.25-1+rpt1 (2025-04-30) aarch64 GNU/Linux

Logs

May 27 11:59:10 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* Resetting GPU for hang.
May 27 11:59:10 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* V3D_ERR_STAT: 0x00001000
May 27 11:59:11 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* Resetting GPU for hang.
May 27 11:59:11 rzx kernel: v3d fec00000.v3d: [drm:v3d_reset [v3d]] *ERROR* V3D_ERR_STAT: 0x00001000

Additional context

I can stable reproduce this issue on Octave 10.1.0 build from official sources. But since original issue also was reproduced on Octave 9.4.0, I think it should be reproducible on 9.4.0 and older versions of octave either, but I didn't tested it with older versions.

you can build and install octave in the following way:

# prerequisites
sudo apt-get install gcc g++ gfortran make libopenblas-dev liblapack-dev libpcre3-dev libarpack2-dev libcurl4-gnutls-dev epstool libfftw3-dev fig2dev libfltk1.3-dev libfontconfig1-dev libfreetype6-dev libgl2ps-dev libglpk-dev libreadline-dev gnuplot libgraphicsmagick++1-dev libhdf5-dev libsndfile1-dev llvm-dev texinfo libgl1-mesa-dev libosmesa6-dev pstoedit portaudio19-dev libjack-jackd2-dev libqhull-dev libqrupdate-dev libqt5core5a qtbase5-dev qttools5-dev qttools5-dev-tools libqscintilla2-qt5-dev libsuitesparse-dev texlive texlive-latex-extra libxft-dev zlib1g-dev autoconf automake bison flex gperf gzip icoutils librsvg2-bin libtool perl rsync tar libsundials-dev git rapidjson-dev
sudo apt-get install openjdk-17-jdk

# download source, build and install
wget https://ftp.gnu.org/gnu/octave/octave-10.1.0.tar.gz
tar -xzf octave-10.1.0.tar.gz 
cd octave-10.1.0

mkdir .build && cd .build && ./../configure
make -j2
sudo make install

I may miss some prerequisite library, so if something is missing just add.