openwrt / openwrt/mt76

MT7915e: causing random kernel OOPS and panic

Open
#972 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
888
Forks
436
PR merge metrics
No merged PRs in 30d

Description

The device experiences random crashes, occurring approximately once a week, particularly when handling 50+ wireless devices.

Device: JDCloud RE-CP-03
OpenWRT version: 24.10.0
MT76 driver version:

PKG_SOURCE_URL:=https://github.com/openwrt/mt76
PKG_SOURCE_PROTO:=git
PKG_SOURCE_DATE:=2025-01-14
PKG_SOURCE_VERSION:=8e4f72b682e9070108536507c5e2720b18c3816d
PKG_MIRROR_HASH:=fa8c5a2ece9e7287605910d9f906b601711c7863613addaadd666f9e3858a9e7

dmesg-ramoops-0 log:

Oops#1 Part1
<7>[126376.343387] sp : ffffffc081873c00
<7>[126376.346772] x29: ffffffc081873c00 x28: 000000000000003c x27: ffffff80066d4130
<7>[126376.353975] x26: 0000000000000000 x25: 0000000000000000 x24: ffffff80084a58b0
<7>[126376.361178] x23: 0000000000000100 x22: 0000000000020000 x21: ffffff80066d1fe0
<7>[126376.368380] x20: 0000000000000016 x19: 0000000000000000 x18: ffffffffffffc800
<7>[126376.375583] x17: 0000000000006998 x16: 0000000000003fb8 x15: 0000000000002788
<7>[126376.382786] x14: 0000000000000004 x13: ffffff80066d6228 x12: 0000000000000000
<7>[126376.389989] x11: 0000000000000040 x10: ffffff80066d6230 x9 : ffffff80066d6228
<7>[126376.397191] x8 : ffffff8024619940 x7 : 0000000000000000 x6 : 0000000000000000
<7>[126376.404393] x5 : ffffff8024619918 x4 : ffffff8024619940 x3 : 000000000000001f
<7>[126376.411595] x2 : ffffff800108e800 x1 : 0000000000000000 x0 : ffffff80084a58b0
<7>[126376.418797] Call trace:
<7>[126376.421316]  mt7915_mac_wtbl_lmac_addr+0x7dc/0x8fc [mt7915e]
<7>[126376.427051]  mt7915_rx_check+0x2c/0xc8 [mt7915e]
<7>[126376.431743]  mt76_dma_rx_poll+0x410/0xc20 [mt76]
<7>[126376.436437]  __napi_poll+0x34/0x1b8
<7>[126376.440003]  napi_threaded_poll_loop+0x1bc/0x1e4
<7>[126376.444691]  napi_threaded_poll+0x70/0x7c
<7>[126376.448772]  kthread+0xd8/0xdc
<7>[126376.451899]  ret_from_fork+0x10/0x20
<4>[126376.455550] ---[ end trace 0000000000000000 ]---
<6>[126376.557611] ieee80211 phy0: WA: free done event
<6>[126376.557611] 10225e40
<6>[126376.557611] len = 40
<6>[126376.557611] DW0 : 28 00 07 30
<6>[126376.557611] DW1 : 02 d8 84 5c
<6>[126376.557611] DW2 : 00 80 01 81
<6>[126376.557611] DW3 : 2c 00 00 41
<6>[126376.557611] DW4 : 06 10 06 08
<6>[126376.557611] DW5 : 0e 90 07 08
<6>[126376.557611] DW6 : 1b 10 01 00
<6>[126376.557611] DW7 : 00 00 02 81
<6>[126376.557611] DW8 : 0b 00 00 41
<6>[126376.557611] DW9 : 05 80 ff 3f
<6>[126376.719854] ieee80211 phy0: WA: free done event
<6>[126376.719854] 10228a00
<6>[126376.719854] len = 20
<6>[126376.719854] DW0 : 14 00 01 30
<6>[126376.719854] DW1 : 01 99 84 4f
<6>[126376.719854] DW2 : 00 40 03 89
<6>[126376.719854] DW3 : 00 00 00 41
<6>[126376.719854] DW4 : 06 90 ff 3f
<6>[126447.923917] ieee80211 phy0: WA: free done event
<6>[126447.923917] 10227740
<6>[126447.923917] len = 20
<6>[126447.923917] DW0 : 14 00 01 30
<6>[126447.923917] DW1 : 01 a7 c4 23
<6>[126447.923917] DW2 : 00 c0 08 89
<6>[126447.923917] DW3 : 31 30 00 41
<6>[126447.923917] DW4 : 04 80 ff 3f
<1>[126486.481324] Unable to handle kernel paging request at virtual address 15f45da9cca9dc63
<1>[126486.489337] Mem abort info:
<1>[126486.492235]   ESR = 0x0000000096000004
<1>[126486.496072]   EC = 0x25: DABT (current EL), IL = 32 bits
<1>[126486.501466]   SET = 0, FnV = 0
<1>[126486.504600]   EA = 0, S1PTW = 0
<1>[126486.507812]   FSC = 0x04: level 0 translation fault
<1>[126486.512773] Data abort info:
<1>[126486.515724]   ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
<1>[126486.521285]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
<1>[126486.526418]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
<1>[126486.531807] [15f45da9cca9dc63] address between user and kernel address ranges
<0>[126486.539015] Internal error: Oops: 0000000096000004 [#1] SMP
<7>[126486.544657] Modules linked in: pppoe ppp_async nft_fib_inet nf_flow_table_inet wireguard pppox ppp_mppe ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_fullcone(O) nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_flow_table nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_sane nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast nf_conntrack_amanda nf_conntrack mt7915e(O) mt76_connac_lib(O) mt76(O) mac80211(O) libchacha20poly1305 chacha_neon cfg80211(O) ts_kmp ts_fsm ts_bm tcp_bbr slhc poly1305_neon nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 macvlan libcurve25519_generic libcrc32c libchacha compat(O) cls_flower asn1_decoder act_vlan
<7>[126486.544809]  crypto_safexcel cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact ip6_udp_tunnel udp_tunnel xsk_diag netlink_diag veth tun crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha512_arm64 sha1_ce sha1_generic seqiv md5 geniv des_generic libdes authencesn authenc arc4 leds_gpio gpio_button_hotplug(O) aquantia
<7>[126486.672805] CPU: 0 PID: 2825 Comm: tailscaled Tainted: G        W  O       6.6.73 #0
<7>[126486.680614] Hardware name: JDCloud RE-CP-03 (DT)
<7>[126486.685299] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
<7>[126486.692327] pc : kmem_cache_alloc_node+0xb4/0x2bc
<7>[126486.697109] lr : kmem_cache_alloc_node+0x40/0x2bc
<7>[126486.701884] sp : ffffffc084c83a80
<7>[126486.705269] x29: ffffffc084c83a80 x28: 0000000000000003 x27: 0000000000000000
<7>[126486.712471] x26: 0000000000000cc0 x25: 00000000000000e0 x24: 00000000ffffffff
<7>[126486.719673] x23: ffffffc08075ad48 x22: 0000000000000cc0 x21: 0000000000000000
<7>[126486.726876] x20: ffffffc0810fe000 x19: ffffff80000b6400 x18: 0000000000000000
<7>[126486.734078] x17: 0000000000000000 x16: 0000000000000000 x15: 0000004003468006
<7>[126486.741280] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
<7>[126486.748483] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
<7>[126486.755685] x8 : ffffff800c454218 x7 : 0000000000000000 x6 : ffffffc084c83cd8
<7>[126486.762887] x5 : 2745a7832640340f x4 : 000000000061140d x3 : 63dca9cca95df415
<7>[126486.770089] x2 : 000000000061140c x1 : 0000000000000070 x0 : 15f45da9cca9dbf3
<7>[126486.777293] Call trace:
<7>[126486.779811]  kmem_cache_alloc_node+0xb4/0x2bc
<7>[126486.784239]  __alloc_skb+0x110/0x140
<7>[126486.787891]  alloc_skb_with_frags+0x4c/0x1d0
<7>[126486.792231]  sock_alloc_send_pskb+0x1ec/0x23c
<7>[126486.796659]  tun_ptr_free+0x4b60/0x6c8c [tun]
<7>[126486.801091]  tun_ptr_free+0x57d0/0x6c8c [tun]
<7>[126486.805519]  vfs_write+0x198/0x350
<7>[126486.808993]  ksys_write+0x58/0xd4
<7>[126486.812379]  __arm64_sys_write+0x18/0x20
<7>[126486.816372]  invoke_syscall.constprop.0+0x4c/0xe0
<7>[126486.821147]  do_el0_svc+0x3c/0xbc
<7>[126486.824534]  el0_svc+0x18/0x4c
<7>[126486.827663]  el0t_64_sync_handler+0x118/0x124
<7>[126486.832091]  el0t_64_sync+0x150/0x154
<0>[126486.835828] Code: b9402a61 f9405e65 8b010003 dac00c63 (f8616801) 
<4>[126486.841988] ---[ end trace 0000000000000000 ]---

dmesg-ramoops-1 log:

Panic#2 Part1
<7>[126376.375583] x17: 0000000000006998 x16: 0000000000003fb8 x15: 0000000000002788
<7>[126376.382786] x14: 0000000000000004 x13: ffffff80066d6228 x12: 0000000000000000
<7>[126376.389989] x11: 0000000000000040 x10: ffffff80066d6230 x9 : ffffff80066d6228
<7>[126376.397191] x8 : ffffff8024619940 x7 : 0000000000000000 x6 : 0000000000000000
<7>[126376.404393] x5 : ffffff8024619918 x4 : ffffff8024619940 x3 : 000000000000001f
<7>[126376.411595] x2 : ffffff800108e800 x1 : 0000000000000000 x0 : ffffff80084a58b0
<7>[126376.418797] Call trace:
<7>[126376.421316]  mt7915_mac_wtbl_lmac_addr+0x7dc/0x8fc [mt7915e]
<7>[126376.427051]  mt7915_rx_check+0x2c/0xc8 [mt7915e]
<7>[126376.431743]  mt76_dma_rx_poll+0x410/0xc20 [mt76]
<7>[126376.436437]  __napi_poll+0x34/0x1b8
<7>[126376.440003]  napi_threaded_poll_loop+0x1bc/0x1e4
<7>[126376.444691]  napi_threaded_poll+0x70/0x7c
<7>[126376.448772]  kthread+0xd8/0xdc
<7>[126376.451899]  ret_from_fork+0x10/0x20
<4>[126376.455550] ---[ end trace 0000000000000000 ]---
<6>[126376.557611] ieee80211 phy0: WA: free done event
<6>[126376.557611] 10225e40
<6>[126376.557611] len = 40
<6>[126376.557611] DW0 : 28 00 07 30
<6>[126376.557611] DW1 : 02 d8 84 5c
<6>[126376.557611] DW2 : 00 80 01 81
<6>[126376.557611] DW3 : 2c 00 00 41
<6>[126376.557611] DW4 : 06 10 06 08
<6>[126376.557611] DW5 : 0e 90 07 08
<6>[126376.557611] DW6 : 1b 10 01 00
<6>[126376.557611] DW7 : 00 00 02 81
<6>[126376.557611] DW8 : 0b 00 00 41
<6>[126376.557611] DW9 : 05 80 ff 3f
<6>[126376.719854] ieee80211 phy0: WA: free done event
<6>[126376.719854] 10228a00
<6>[126376.719854] len = 20
<6>[126376.719854] DW0 : 14 00 01 30
<6>[126376.719854] DW1 : 01 99 84 4f
<6>[126376.719854] DW2 : 00 40 03 89
<6>[126376.719854] DW3 : 00 00 00 41
<6>[126376.719854] DW4 : 06 90 ff 3f
<6>[126447.923917] ieee80211 phy0: WA: free done event
<6>[126447.923917] 10227740
<6>[126447.923917] len = 20
<6>[126447.923917] DW0 : 14 00 01 30
<6>[126447.923917] DW1 : 01 a7 c4 23
<6>[126447.923917] DW2 : 00 c0 08 89
<6>[126447.923917] DW3 : 31 30 00 41
<6>[126447.923917] DW4 : 04 80 ff 3f
<1>[126486.481324] Unable to handle kernel paging request at virtual address 15f45da9cca9dc63
<1>[126486.489337] Mem abort info:
<1>[126486.492235]   ESR = 0x0000000096000004
<1>[126486.496072]   EC = 0x25: DABT (current EL), IL = 32 bits
<1>[126486.501466]   SET = 0, FnV = 0
<1>[126486.504600]   EA = 0, S1PTW = 0
<1>[126486.507812]   FSC = 0x04: level 0 translation fault
<1>[126486.512773] Data abort info:
<1>[126486.515724]   ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
<1>[126486.521285]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
<1>[126486.526418]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
<1>[126486.531807] [15f45da9cca9dc63] address between user and kernel address ranges
<0>[126486.539015] Internal error: Oops: 0000000096000004 [#1] SMP
<7>[126486.544657] Modules linked in: pppoe ppp_async nft_fib_inet nf_flow_table_inet wireguard pppox ppp_mppe ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_fullcone(O) nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat_tftp nf_nat_snmp_basic nf_nat_sip nf_nat_pptp nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_flow_table nf_conntrack_tftp nf_conntrack_snmp nf_conntrack_sip nf_conntrack_sane nf_conntrack_pptp nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp nf_conntrack_broadcast nf_conntrack_amanda nf_conntrack mt7915e(O) mt76_connac_lib(O) mt76(O) mac80211(O) libchacha20poly1305 chacha_neon cfg80211(O) ts_kmp ts_fsm ts_bm tcp_bbr slhc poly1305_neon nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 macvlan libcurve25519_generic libcrc32c libchacha compat(O) cls_flower asn1_decoder act_vlan
<7>[126486.544809]  crypto_safexcel cls_bpf act_bpf sch_tbf sch_ingress sch_htb sch_hfsc em_u32 cls_u32 cls_route cls_matchall cls_fw cls_flow cls_basic act_skbedit act_mirred act_gact ip6_udp_tunnel udp_tunnel xsk_diag netlink_diag veth tun crypto_user algif_skcipher algif_rng algif_hash algif_aead af_alg sha512_arm64 sha1_ce sha1_generic seqiv md5 geniv des_generic libdes authencesn authenc arc4 leds_gpio gpio_button_hotplug(O) aquantia
<7>[126486.672805] CPU: 0 PID: 2825 Comm: tailscaled Tainted: G        W  O       6.6.73 #0
<7>[126486.680614] Hardware name: JDCloud RE-CP-03 (DT)
<7>[126486.685299] pstate: 20400005 (nzCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
<7>[126486.692327] pc : kmem_cache_alloc_node+0xb4/0x2bc
<7>[126486.697109] lr : kmem_cache_alloc_node+0x40/0x2bc
<7>[126486.701884] sp : ffffffc084c83a80
<7>[126486.705269] x29: ffffffc084c83a80 x28: 0000000000000003 x27: 0000000000000000
<7>[126486.712471] x26: 0000000000000cc0 x25: 00000000000000e0 x24: 00000000ffffffff
<7>[126486.719673] x23: ffffffc08075ad48 x22: 0000000000000cc0 x21: 0000000000000000
<7>[126486.726876] x20: ffffffc0810fe000 x19: ffffff80000b6400 x18: 0000000000000000
<7>[126486.734078] x17: 0000000000000000 x16: 0000000000000000 x15: 0000004003468006
<7>[126486.741280] x14: 0000000000000000 x13: 0000000000000000 x12: 0000000000000000
<7>[126486.748483] x11: 0000000000000000 x10: 0000000000000000 x9 : 0000000000000000
<7>[126486.755685] x8 : ffffff800c454218 x7 : 0000000000000000 x6 : ffffffc084c83cd8
<7>[126486.762887] x5 : 2745a7832640340f x4 : 000000000061140d x3 : 63dca9cca95df415
<7>[126486.770089] x2 : 000000000061140c x1 : 0000000000000070 x0 : 15f45da9cca9dbf3
<7>[126486.777293] Call trace:
<7>[126486.779811]  kmem_cache_alloc_node+0xb4/0x2bc
<7>[126486.784239]  __alloc_skb+0x110/0x140
<7>[126486.787891]  alloc_skb_with_frags+0x4c/0x1d0
<7>[126486.792231]  sock_alloc_send_pskb+0x1ec/0x23c
<7>[126486.796659]  tun_ptr_free+0x4b60/0x6c8c [tun]
<7>[126486.801091]  tun_ptr_free+0x57d0/0x6c8c [tun]
<7>[126486.805519]  vfs_write+0x198/0x350
<7>[126486.808993]  ksys_write+0x58/0xd4
<7>[126486.812379]  __arm64_sys_write+0x18/0x20
<7>[126486.816372]  invoke_syscall.constprop.0+0x4c/0xe0
<7>[126486.821147]  do_el0_svc+0x3c/0xbc
<7>[126486.824534]  el0_svc+0x18/0x4c
<7>[126486.827663]  el0t_64_sync_handler+0x118/0x124
<7>[126486.832091]  el0t_64_sync+0x150/0x154
<0>[126486.835828] Code: b9402a61 f9405e65 8b010003 dac00c63 (f8616801) 
<4>[126486.841988] ---[ end trace 0000000000000000 ]---
<3>[126486.850773] pstore: backend (ramoops) writing error (-28)
<0>[126486.856244] Kernel panic - not syncing: Oops: Fatal exception
<2>[126486.862056] SMP: stopping secondary CPUs
<0>[126486.866052] Kernel Offset: disabled
<0>[126486.869610] CPU features: 0x0,00000000,00000000,1000400b
<0>[126486.874991] Memory Limit: none

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the mt7915_mac_wtbl_lmac_addr and mt7915_rx_check entry points named in the call trace, alongside the supplied MT76 revision and kernel logs. Reproduce the crash under the reported 50+ wireless-device workload and determine why the driver leads to the kernel paging fault; done means the failure is fixed and the workload no longer produces an OOPS or panic.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
networking, operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.