SignalR per-invocation allocations

I'm building a game that requires low latency and there's some SignalR overhead (that's pretty low tbh) but impossible to get rid of. The game I'm building is sending LOTS of messages (60 FPS) so these are now the top allocations remaining.

Some of them we can't remove, we might be able to pool objects but others we should leave alone. The ones I am interested in improving:
- System.Object\[\]
- Microsoft.AspNetCore.SignalR.ClientProxyExtensions.SendAsync(Microsoft.AspNetCore.SignalR.IClientProxy, string, object, System.Threading.CancellationToken)
- Microsoft.AspNetCore.SignalR.Protocol.JsonHubProtocol.BindTypes(ref System.Text.Json.Utf8JsonReader, System.Collections.Generic.IReadOnlyList)
- Microsoft.AspNetCore.SignalR.Internal.DefaultHubActivator\
- string allocations (https://github.com/dotnet/aspnetcore/issues/41342)
- System.Collections.Concurrent.ConcurrentDictionary<,>.Enumerator (https://github.com/dotnet/runtime/issues/25448)
- Microsoft.AspNetCore.SignalR.DefaultHubLifetimeManager.SendToAllConnections(string, System.Object[], System.Func, object, System.Threading.CancellationToken)
- System.Byte\[\]
- Microsoft.AspNetCore.Internal.MemoryBufferWriter.ToArray\(\)
- Microsoft.AspNetCore.SignalR.Protocol.HubProtocolExtensions.GetMessageBytes\(Microsoft.AspNetCore.SignalR.Protocol.IHubProtocol, Microsoft.AspNetCore.SignalR.Protocol.HubMessage\)
- Microsoft.AspNetCore.SignalR.SerializedHubMessage.GetSerializedMessage\(Microsoft.AspNetCore.SignalR.Protocol.IHubProtocol\)
- Microsoft.AspNetCore.SignalR.SerializedHubMessage
- System.Collections.Generic.List\ - Seems like we allocate a list of empty stream ids since it's always in the payload even when it's empty. (https://github.com/dotnet/aspnetcore/pull/41344)
- There are 3 state machine boxes that allocate per operation (read/write)
- Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.ReadAsyncInternal() (https://github.com/dotnet/aspnetcore/pull/41345)
- System.IO.Pipelines.StreamPipeReader.ReadAsync.__Core|36_0() (https://github.com/dotnet/runtime/pull/68457)
- System.Net.Security.SslStream.ReadAsyncInternal() https://github.com/dotnet/runtime/issues/68467

|Type|Allocations|Bytes|Average Size \(Bytes\)|
|-|-|-|-|
| - System.Runtime.CompilerServices.AsyncTaskMethodBuilder\<\>.AsyncStateMachineBox\<\>|2,418|477,152|197.33|
| - System.Object\[\]|1,659|53,928|32.51|
| - Microsoft.AspNetCore.SignalR.Protocol.InvocationMessage|1,631|91,336|56|
| - System.String|1,629|54,420|33.41|
| - System.Byte\[\]|862|302,374|350.78|
| - System.Collections.Concurrent.ConcurrentDictionary\<,\>.Enumerator|858|55,040|64.15|
| - Microsoft.AspNetCore.SignalR.SerializedHubMessage|825|72,600|88|
| - System.Object|825|19,800|24|
| - System.Int32\[\]|808|29,196|36.13|
| - System.Collections.Generic.List\<\>|808|25,856|32|
| - System.Collections.Generic.Dictionary\<,\>.Entry\[\]|806|96,720|120|
| - System.Collections.Generic.Dictionary\<,\>|806|64,480|80|
| - BombRMan.Hubs.GameServer|806|45,136|56|
| - Microsoft.AspNetCore.SignalR.Protocol.CompletionMessage|806|45,136|56|
| - Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope|806|38,688|48|
| - Microsoft.AspNetCore.SignalR.Internal.DefaultHubActivator\|806|25,792|32|