oxidecomputer / oxidecomputer/hubris
Smarter memory allocation for RAMs of differing capability
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 239
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 23
Description
Right now, the Hubris build system basically thinks in terms of two RAMs:
- A big one, called
ram, where task stacks and memory go. - Additional user-defined RAMs that must be mapped explicitly, like our use of STM32H7 SRAM3 for Ethernet DMA and packet storage.
This isn't ideal, because in general, SoCs in this class have multiple RAMs at discontiguous locations with differing capabilities. Consider the STM32H753, a particularly egregious case:
- 64 kiB of ITCM, suitable primarily for code and vector tables. Dedicated 64-bit bus. You can put data in this RAM but, with the way it's wired to the processor, you probably don't want to.
- 128 kiB of DTCM, a fast data-oriented zero-wait-state memory. Dedicated 64-bit bus. This is where we've historically mapped all of Hubris.
- 512 kiB of AXI SRAM, a reasonably fast (but slower than TCM) large memory. Shared 64-bit bus. This is where I've proposed moving Gimlet's RAM in #717 to unblock development there.
- 128 kiB "SRAM1," on a shared 32-bit bus reachable indirectly through the AXI fabric.
- 128 kiB "SRAM2," similar
- 32 kiB "SRAM3," similar - currently used for Ethernet data, because the Ethernet MAC hardware has direct access to the 32-bit AHB fabric where this lives
(ST have apparently gotten the message that this is annoying as heck because their newer parts, particularly the STM32H7A3 line, advertise having all their RAM at contiguous addresses. Yay. Unfortunately they don't have Ethernet so we can't use them.)
We will want a clever-er way of mapping RAM than what we currently do, if we want to be able to use all the RAM on parts like this. One possible decision for STM32H753 would be:
- Kernel state and stack go in DTCM for zero-wait-state access, to maintain system call performance.
- Task stacks also go in DTCM, to keep zero-wait-state task stack manipulation.
- Task data/bss go in the large AXI SRAM.
- Ethernet buffers go in SRAM3 (we do this today).
- All ringbufs go somewhere inconveniently out of the way, such as SRAM3 or SRAM2.
The main technical changes required to do something like this are:
- Don't assume the name
ramfor where all the memory-things go. - Allow kernel and task data to be routed to RAMs separately. (Ideally, allow individual tasks to override this.)
- Allow task stack and task data to be discontiguous. (Currently stacks are always at the bottom of task RAM.)
As we add additional discontiguous segments, we burn MPU slots. In most cases, this is fine -- tasks typically use only 3-4 MPU slots at present, and we have 8 to work with. But that imposes an upper bound on just how ridiculous we can go with discontiguous RAM.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name files or tests. Start by tracing how the build system currently assigns ram and lays out kernel state, task stacks, task data/bss, and ringbufs; then determine where those mappings are represented. Done means the system can route these categories, and ideally individual tasks, across discontiguous RAMs without relying on one ram region.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- build-system, embedded-iot, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100