adafruit / adafruit/Adafruit_CircuitPython_HTTPServer
minimizing GC allocations with Websocket.send_message
- Dominant language
- Python
- Stars
- 52
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
## Motivation
Every call to **Websocket.sendMessage(...)** appears to be allocating memory which will hang around until the next GC cycle. I'm checking directly before/after the call, i.e.
```python
gc_after_message_create = gc.mem_free()
self.websocket.send_message(message, fail_silently=True)
gc_after = gc.mem_free()
```
so it's not including anything due to the creation/formatting of **message**.
Sending a 29 character JSON packet leaves 112 bytes allocated. A 31 or 32 character packet leaves 128 bytes.
If there's much traffic, these can add up quickly (especially considering that there's also the allocation overhead for the message passed in and artifacts of it's formatting) and force more frequent gc collection cycles.
This isn't a simple thing to "fix", AFAIK there may be significant limitations on what can be done with the current API. However, some/most/maybe all of the leftover allocations could potentially be eliminated by having a separate object managing all the message bits, which could then be passed in to a new method (or "overloaded" on send_message with an internal isinstance check)
## Current behavior
gc "leaks" (leftovers per invocation of send_message until next **gc.collect()**) include
- passing message as **str | bytes** - both require an immutable per-message allocation before you can even call **send_message(...)**
- ```message.encode()```* for str/text messages - likely creates a copy
- in ```def _prepare_frame(opcode: int, message: bytes) -> bytearray:```
- creating new **bytearray** for frame - another copy
- ```payload_length.to_bytes(2, "big")``` and ```payload_length.to_bytes(8, "big")``` both leave allocations (32 bytes)
- but ```frame.append( (payload_length>>24) & 0xFF ) ...```does not
- _ISocket
- ```def send(self, data: bytes) -> int: ...`` - if this actually uses ore creates data as bytes internally (not just a **memoryview**), that's potentially another copy
The three potential internal copies (message encode, frame bytearray, _Isocket.send) plus the leftovers from payload_length.to_bytes(...) appear to match fairly closely with my measured results (on an admittedly small sample set).
## Potential solutions
Eliminating this overhead (probably???) requires some form of reusable mutable buffer which ideally can be instantiated and passed in by the caller.
Might be good to wrap this buffer as a member of an object which can handle things like synchronization (don't change it until it's done writing...). Possibilities for the buffer include
- ```io.StringIO``` / ```io.BinIO```
- can be _partially_ reset using ```instance.seek(0)```
- but unfortunately
- ```instance.truncate()``` is not implemented, and even if it were it only "works" if it holds and reuses allocated memory (like calling **clear()** on a C++ std::vector but not calling **shrink_to_fit()** - "resets" the array but retains the underlying memory / capacity )
- doesn't seem to provide access to the underlying data except through copy (**getValue()**) i.e. no **memoryview**
- ```bytearray```
- not resettable, no way to "rewind" / seek(0)
- not shrinkable
- CircuitPython bytearray does not support slice assignment
- in in regular CPython you can use ```del byteArrayInstace[:]```
- same caveat as truncate above - only "helps" if it holds memory
- ```ulab.numpy.ndarray(...,dtype=uint8)```
- also not shrinkable
Whatever is used needs a way to pass the buffer data without reallocation to whatever is actually implementing _ISocket.send. It seems that the minimum requirement would be **memorview** / readable buffer protocol support. There might also need to be some form of notification of send failed/complete unless **_ISocket.send(...)** is a fully blocking implementation (in which case, there might be another feature request before long...)
I suppose this (probably more than) enough to get the conversation started...
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing Websocket.send_message through _prepare_frame and the _ISocket.send interface, focusing on the allocations described in the issue. Determine whether a reusable buffer and memoryview-compatible send path are feasible; done would require an agreed API and measurements showing reduced per-message allocations without breaking send completion or failure handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100