Difference in PCIe CLKREQ# behavior between CM4 Rev 1.0 and Rev 1.1

Start with drivers/pci/controller/pcie-brcmstb.c and the linked upstream Linux commit, then compare the CLKREQ# behavior and logs from CM4 Rev 1.0 and Rev 1.1 using the reported setup. Done means confirming whether the revision difference is hardware-related or requires a kernel change, with enough evidence for maintainers to respond.

Describe the bug

First, I was sent here after posting on the RPi Forum here. I am trying to upstream Linux a commit for pcie-brcmstb.c wrt handling CLKREQ#. My goal is to be backwards-compatible with the RPi DT convention of "brcm,enable-l1ss" which was never upstreamed, while providing coverage of devices with a missing/floating CLKREQ# pin w/o forcing L1SS mode.

My commits seem to work fine on a CM4 Rev 1.1 module, but fail on a CM4 Rev 1.0 module, all using the same endpoint device and IO board. I am hoping if you can tell me if there are differences between the two Revisions regarding how the CLKREQ# signal is handled.

Regards,
Jim Quinlan
Broadcom STB

Steps to reproduce the behaviour

This is going to be difficult because the test is the latest upstream linux with this commit applied. I can provide further details but I am hoping that someone in the know can answer my question of the top of their head.

Device (s)

Raspberry Pi CM4

System

Upstream Linux with my aforementioned commit, version ~6.3.0-rc5.
I do not think the FW version is relevant since the difference is only observed between Rev 1.0 and Rev 1.1 of the CM4.

Logs

This is all the tester posted for the CM4 Rev 1.0 test. The CM4 Rev 1.1 test works fine.

1.729490] brcm-pcie fd500000.pcie: host bridge /scb/pcie@7d500000 ranges:

[ 1.736532] brcm-pcie fd500000.pcie: No bus range found for /scb/pcie@7d500000, using [bus 00-ff]
[ 1.745686] brcm-pcie fd500000.pcie: MEM 0x0600000000..0x0603ffffff -> 0x00f8000000
[ 1.753873] brcm-pcie fd500000.pcie: IB MEM 0x0000000000..0x01ffffffff -> 0x0400000000
[ 1.762081] brcm-pcie fd500000.pcie: forcing PERST# to be asserted ← local patch
[ 1.768933] brcm-pcie fd500000.pcie: PCI host bridge to bus 0000:00
[ 1.775275] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 1.780818] pci_bus 0000:00: root bus resource [mem 0x600000000-0x603ffffff] (bus address [0xf8000000-0xfbffffff])
[ 1.791297] pci 0000:00:00.0: [14e4:2711] type 01 class 0x060400
[ 1.797414] pci 0000:00:00.0: PME# supported from D0 D3hot
[ 1.909851] brcm-pcie fd500000.pcie: using brcm,completion-timeout-us = 1000000 ← local patch
[ 1.917241] brcm-pcie fd500000.pcie: uni-dir CLKREQ# for L0s, L1 ASPM
[ 1.925780] brcm-pcie fd500000.pcie: link up, 5.0 GT/s PCIe x1 (SSC)
[ 1.980345] SError Interrupt on CPU0, code 0x00000000bf000002 -- SError
[ 1.980350] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.3.0-rc5+ #3
[ 1.980355] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT)
[ 1.980357] pstate: 200000c5 (nzCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 1.980362] pc : pci_generic_config_read+0x44/0xe8
[ 1.980378] lr : pci_generic_config_read+0x2c/0xe8
[ 1.980383] sp : ffff80000802b800
[ 1.980385] x29: ffff80000802b800 x28: 0000000000000000 x27: 0000000000000001
[ 1.980393] x26: ffff3f2000bcb000 x25: 0000000000000001 x24: 0000000000000000
[ 1.980399] x23: ffff80000802b904 x22: 0000000000000000 x21: ffffa270708da000
[ 1.980404] x20: 0000000000000004 x19: ffff80000802b864 x18: 0000000000000000
[ 1.980410] x17: 30303031203d2073 x16: 752d74756f656d69 x15: 0000000000000030
[ 1.980415] x14: 0000000000000000 x13: 2943535328203178 x12: 206549435020732f
[ 1.980420] x11: 00000000ffffefff x10: 00000000ffffefff x9 : ffffa2706f0e2834
[ 1.980426] x8 : 0000000000017fe8 x7 : c0000000ffffefff x6 : 000000000000000b
[ 1.980431] x5 : ffff3f2000bcbb80 x4 : ffff800008a20000 x3 : ffff800008a20000
[ 1.980436] x2 : 0000000000008000 x1 : 00000000deaddead x0 : ffff800008a28000
[ 1.980443] Kernel panic - not syncing: Asynchronous SError Interrupt
[ 1.980445] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.3.0-rc5+ #3
[ 1.980449] Hardware name: Raspberry Pi Compute Module 4 Rev 1.0 (DT)
[ 1.980452] Call trace:
[ 1.980454] dump_backtrace+0xa8/0x138
[ 1.980460] show_stack+0x20/0x38
[ 1.980463] dump_stack_lvl+0x48/0x60
[ 1.980471] dump_stack+0x18/0x28
[ 1.980476] panic+0x378/0x398
[ 1.980482] nmi_panic+0xb4/0xc0
[ 1.980486] arm64_serror_panic+0x78/0x90
[ 1.980490] do_serror+0x30/0x70
[ 1.980494] el1h_64_error_handler+0x30/0x48
[ 1.980498] el1h_64_error+0x64/0x68
[ 1.980502] pci_generic_config_read+0x44/0xe8
[ 1.980507] pci_bus_read_config_dword+0x98/0x140
[ 1.980513] pci_bus_generic_read_dev_vendor_id+0x3c/0x1c0
[ 1.980517] pci_scan_single_device+0xa8/0x118
[ 1.980520] pci_scan_slot+0x6c/0x1e0
[ 1.980523] pci_scan_child_bus_extend+0x50/0x2e0
[ 1.980527] pci_scan_bridge_extend+0x15c/0x5a8
[ 1.980530] pci_scan_child_bus_extend+0x110/0x2e0
[ 1.980534] pci_scan_root_bus_bridge+0x6c/0xf8
[ 1.980537] pci_host_probe+0x20/0xd0
[ 1.980541] brcm_pcie_probe+0x294/0x620
[ 1.980546] platform_probe+0x70/0xe8
[ 1.980553] really_probe+0x18c/0x3d8
[ 1.980557] __driver_probe_device+0x84/0x198
[ 1.980561] driver_probe_device+0x44/0x120
[ 1.980565] __driver_attach+0xfc/0x210
[ 1.980569] bus_for_each_dev+0x7c/0xe8
[ 1.980572] driver_attach+0x2c/0x40
[ 1.980576] bus_add_driver+0x118/0x228
[ 1.980580] driver_register+0x68/0x138
[ 1.980584] __platform_driver_register+0x30/0x48
[ 1.980589] brcm_pcie_driver_init+0x24/0x38
[ 1.980595] do_one_initcall+0x4c/0x238
[ 1.980599] kernel_init_freeable+0x21c/0x3f0
[ 1.980605] kernel_init+0x2c/0x1f8
[ 1.980609] ret_from_fork+0x10/0x20
[ 1.980615] SMP: stopping secondary CPUs
[ 1.980622] Kernel Offset: 0x227066a00000 from 0xffff800008000000
[ 1.980624] PHYS_OFFSET: 0xffffc0e100000000
[ 1.980626] CPU features: 0x400002,013c0400,0000421b
[ 1.980630] Memory Limit: none
[ 2.280719] ---[ end Kernel panic - not syncing: Asynchronous SError Interrupt ]---

Additional context

I have been copying PhilE on my Linux submits -- or at least his emails haven't bounced -- so he should know my email address if anyone wants to communicate directly.