Clean up usage of Linq and `.ToArray()`.

Both Linq and `ToArray` produce allot of Garbage, and should not be used at all unless absolutely necessary.
Using it for loading and saving is ok, but in a game loop no. This is because things like `ToArray` make copies of the data. If these are only used for a short period of time then the garbage collector will need to clean them up. It would be better to use the lists directly.
Also `.Any()` should be removed in favour of `.Length > 0` for exactly the same reason.

We should review all the code and remove usage.