Make object ownership, lifetime and non-nullable pointers more obvious

Follow up from #1671 from discussion with @neobrain

> Better tracking allocation/free responsibilities, as well as nullable pointers would be an interesting code quality improvement.

Myself I'm not much of a fan of C++ references, mostly because of the confusion they can create over what is a pointer and what is not, and I mostly use to rename locals in functions. The codebase mixes in pointers and references in arbitrary ways, which can create confusion.

There's a couple of arguments for using more references, to communicate among other things
- non nullable uses
- pointer constness
- Allocation / Free semantics

We could easily track these with some annotating templates, and avoid references, or we could go full in references and whatnot.

Thoughts?