tinygrad / tinygrad/tinygrad

hcq2 + chestnut broken when `GMMU=1`

Open
#18,136 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
33.6k
Forks
4.3k
Avg merge
3h 11m
Merged PRs (30d)
491

Description

On USB3 with hcq2, we attempt to zero allocate a 4mb buffer with the 0xF0 control message, which wrties at 3.6mb/s (problematic line is here. This obviously fails as the default timeout is 1000ms. Ex running DEV=USB+AMD python3 -m tinygrad.llm --serve 9000 will fail with RuntimeError: bulk OUT 0x02 failed: Operation timed out

Bumping up the timeout to 5000ms on this yields a different issue as writing the 4mb buffer all at once seems to lock up the chestnut from receiving further commands.

On master, the second run of this script will fail and the chestnut will become locked until it is restarted (errors with RuntimeError: libusb_get_string_descriptor_ascii: Operation timed out):

from tinygrad.runtime.support.usb import CustomASM24Controller, USB3
from tinygrad.runtime.support.hcq import hcq_filter_visible_devices
from tinygrad.runtime.support.system import System

devs =  hcq_filter_visible_devices(USB3.list_devices(0xADD1, 0x0001) + USB3.list_devices(0x3801, 0x0001), "AMD")

device_ptr, _ = devs[0]
usb = USB3(device_ptr)
c = CustomASM24Controller(usb)
bars = System.pci_setup_usb_bars(c, gpu_bus=4, mem_base=0x10000000, pref_mem_base=(32 << 30))
vram_base, vram_size = bars[0]
offset, write_size = 0x40100000, 4 << 20
address = vram_base + offset
c._f0_out(0x60, 0x0F, address, len(bytes(write_size)) // 4, mode=1)
c.usb.bulk_write(bytes(write_size), 5000)
print("Success!")

A workaround (?) here is to just set GMMU=0 since that disables the problematic path here.

Is running with GMMU=0 the intended behaviour with chestnut? Its not the default but I see this is set for some of the CI stages...

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure with DEV=USB+AMD python3 -m tinygrad.llm --serve 9000 and the provided USB3 script. Read tinygrad/runtime/support/memory.py around lines 250-256, then inspect the CustomASM24Controller, USB3, and HCQ entry points used by the script. Done means the default GMMU path handles chestnut reliably, including repeated runs, without requiring GMMU=0.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
embedded-iot
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.