openwrt / openwrt/mt76

mt7925/mt792x: module unload + reboot/shutdown hang (rx NAPI double-disable; MT7927 MCU teardown reset storm)

Open
#1,124 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
888
Forks
436
PR merge metrics
No merged PRs in 30d

Description

Summary

On an MT7927 (Filogic 380, PCI 14c3:7927) running kernel 7.0.0 with mt76 at
b2704cf, modprobe -r mt7925e reliably hangs: the modprobe task is stuck in
D state with the module refcount at -1 and cannot be killed. Because the PCI
.shutdown callback (mt7925_pci_shutdown) also calls mt7925_pci_remove(),
this hangs system reboot and shutdown too.

Bisecting the teardown turned up two independent bugs, both in the shared
mt792x/mt76 teardown path. Fixing only one just moves the hang to the other.

Environment
  • Card: MediaTek MT7927 / Filogic 380, 14c3:7927, chip family MT6639
  • Driver: mt7925e
  • Kernel: 7.0.0 (bug 1 is relevant to any kernel >= 6.15 — see below)
  • mt76: b2704cf

Bug 1 — rx NAPI double-disable deadlock (also affects mt7921)

mt7925e_unregister_device() disables the rx NAPIs:

/* mt7925/pci.c */
mt76_unregister_device(&dev->mt76);
mt76_for_each_q_rx(&dev->mt76, i)
        napi_disable(&dev->mt76.napi[i]);

then, later in the same teardown, mt792x_dma_cleanup() -> mt76_dma_cleanup()
disables the same rx NAPIs a second time:

/* dma.c, mt76_dma_cleanup() */
mt76_for_each_q_rx(dev, i) {
        ...
        napi_disable(&dev->napi[i]);
        netif_napi_del(&dev->napi[i]);
        ...
}

On kernels >= 6.15 napi_disable() leaves NAPI_STATE_SCHED set as the
"disabled" marker, so the second napi_disable() on an already-disabled NAPI
loops forever (nothing will ever clear SCHED). The stuck task:

napi_disable_locked
napi_disable
mt76_dma_cleanup       [mt76]
mt792x_dma_cleanup     [mt792x_lib]
mt7925_pci_remove      [mt7925e]
pci_device_remove
device_release_driver_internal
driver_detach
pci_unregister_driver
mt7925_pci_driver_exit [mt7925e]
__do_sys_delete_module

mt7921e_unregister_device() (mt7921/pci.c) has the identical
mt76_for_each_q_rx napi_disable(&dev->mt76.napi[i]) loop, so mt7921 should hit
the same hang on >= 6.15 kernels.

Suggested fix: drop the redundant napi_disable() loop from
*_unregister_device()mt76_dma_cleanup() already disables and deletes
those NAPIs.


Bug 2 — MT7927 MCU teardown timeout -> reset storm

With bug 1 worked around, teardown then hangs earlier, spamming:

mt7925e 0000:0a:00.0: Message 00020002 (seq 2) timeout
mt7925e 0000:0a:00.0: Message 00020008 (seq 3) timeout
mt7925e 0000:0a:00.0: Message 00020016 (seq 4) timeout
...
wlp10s0: failed to remove key (0, ...) from hardware (-110)

On the MT7927 the MCU stops responding once teardown begins, so every teardown
MCU command times out (~3 s). Worse, on timeout mt7925_mcu_parse_response()
kicks a device reset:

/* mt7925/mcu.c, mt7925_mcu_parse_response() */
if (!skb) {
        dev_err(mdev->dev, "Message %08x (seq %d) timeout\n", cmd, seq);
        mt792x_reset(mdev);          /* queues reset_work */
        return -ETIMEDOUT;
}

mt792x_reset() queues reset_work, which re-inits the device and sends more
MCU commands, which time out, which queue another reset — an unbounded
timeout -> reset -> timeout loop.

MT76_REMOVED is set at the very start of mt7925_pci_remove() and is already
honored in the DMA/IRQ paths (mt792x_dma.c, mt792x_irq_handler), but the MCU
send path does not check it, so teardown MCU commands are still issued and still
(slowly) time out.

Suggested fix (two parts):

  1. Honor MT76_REMOVED in mt76_mcu_skb_send_and_get_msg() — fail MCU sends
    fast during teardown (mirrors the existing bus_hung guard right below it and
    the DMA/IRQ paths).
  2. Skip mt792x_reset() in mt7925_mcu_parse_response() when MT76_REMOVED is
    set — don't kick a reset while the device is being removed.

Result

With both fixed, modprobe -r mt7925e completes cleanly (~2 s) and
reboot/shutdown no longer hang; wifi is unaffected across unload/reload.

Working patches against b2704cf are here for reference:
https://github.com/ardor99/mt7927-wifi7-dkms/tree/main/patches — happy to send
these to the linux-wireless list as proper patches if that is preferred.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with mt7925/pci.c, mt7921/pci.c, dma.c, mt7925/mcu.c, mt792x_dma.c, mt792x_irq_handler, and mt76_mcu_skb_send_and_get_msg(). Trace device removal through both teardown paths and compare the existing MT76_REMOVED and bus_hung guards. Done means unload, reboot, and shutdown complete without NAPI deadlock or reset storms, while unload/reload preserves Wi-Fi behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
networking, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.