Tokenizer should be compliant with ANSI:SQL 2011

This is a large issue that needs to be broken out into smaller issues. The current tokenizer is quite simplistic and needs to be improved to be compliant with section 5 of this specification:

https://jakewheat.github.io/sql-overview/sql-2011-foundation-grammar.html#_5_lexical_elements

Start by reviewing the current tokenizer and the SQL:2011 Foundation Grammar, section 5, linked in the issue. Break the broad compliance work into smaller, scoped issues, each identifying a tokenizer change and a clear completion condition.

Written by the indexing model from the issue text.