adafruit / adafruit/Adafruit_CircuitPython_HTTPServer

minimizing GC allocations with Websocket.send_message

Open
#108 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
52
Forks
30
PR merge metrics
No merged PRs in 30d

Description

## Motivation
Every call to **Websocket.sendMessage(...)** appears to be allocating memory which will hang around until the next GC cycle. I'm checking directly before/after the call, i.e.
```python
gc_after_message_create = gc.mem_free()
self.websocket.send_message(message, fail_silently=True)
gc_after = gc.mem_free()
```
so it's not including anything due to the creation/formatting of **message**.

Sending a 29 character JSON packet leaves 112 bytes allocated. A 31 or 32 character packet leaves 128 bytes.

If there's much traffic, these can add up quickly (especially considering that there's also the allocation overhead for the message passed in and artifacts of it's formatting) and force more frequent gc collection cycles.

This isn't a simple thing to "fix", AFAIK there may be significant limitations on what can be done with the current API. However, some/most/maybe all of the leftover allocations could potentially be eliminated by having a separate object managing all the message bits, which could then be passed in to a new method (or "overloaded" on send_message with an internal isinstance check)

## Current behavior

gc "leaks" (leftovers per invocation of send_message until next **gc.collect()**) include

- passing message as **str | bytes** - both require an immutable per-message allocation before you can even call **send_message(...)**
- ```message.encode()```* for str/text messages - likely creates a copy
- in ```def _prepare_frame(opcode: int, message: bytes) -> bytearray:```
- creating new **bytearray** for frame - another copy
- ```payload_length.to_bytes(2, "big")``` and ```payload_length.to_bytes(8, "big")``` both leave allocations (32 bytes)
- but ```frame.append( (payload_length>>24) & 0xFF ) ...```does not
- _ISocket
- ```def send(self, data: bytes) -> int: ...`` - if this actually uses ore creates data as bytes internally (not just a **memoryview**), that's potentially another copy

The three potential internal copies (message encode, frame bytearray, _Isocket.send) plus the leftovers from payload_length.to_bytes(...) appear to match fairly closely with my measured results (on an admittedly small sample set).

## Potential solutions

Eliminating this overhead (probably???) requires some form of reusable mutable buffer which ideally can be instantiated and passed in by the caller.
Might be good to wrap this buffer as a member of an object which can handle things like synchronization (don't change it until it's done writing...). Possibilities for the buffer include
- ```io.StringIO``` / ```io.BinIO```
- can be _partially_ reset using ```instance.seek(0)```
- but unfortunately
- ```instance.truncate()``` is not implemented, and even if it were it only "works" if it holds and reuses allocated memory (like calling **clear()** on a C++ std::vector but not calling **shrink_to_fit()** - "resets" the array but retains the underlying memory / capacity )
- doesn't seem to provide access to the underlying data except through copy (**getValue()**) i.e. no **memoryview**
- ```bytearray```
- not resettable, no way to "rewind" / seek(0)
- not shrinkable
- CircuitPython bytearray does not support slice assignment
- in in regular CPython you can use ```del byteArrayInstace[:]```
- same caveat as truncate above - only "helps" if it holds memory
- ```ulab.numpy.ndarray(...,dtype=uint8)```
- also not shrinkable

Whatever is used needs a way to pass the buffer data without reallocation to whatever is actually implementing _ISocket.send. It seems that the minimum requirement would be **memorview** / readable buffer protocol support. There might also need to be some form of notification of send failed/complete unless **_ISocket.send(...)** is a fully blocking implementation (in which case, there might be another feature request before long...)

I suppose this (probably more than) enough to get the conversation started...

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing Websocket.send_message through _prepare_frame and the _ISocket.send interface, focusing on the allocations described in the issue. Determine whether a reusable buffer and memoryview-compatible send path are feasible; done would require an agreed API and measurements showing reduced per-message allocations without breaking send completion or failure handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.