Use `tree-sitter-r` to parse the document

Currently, the document is parsed manually in a very inefficient way.

For example, find_unbalanced_bracket painstakingly reads a line byte by byte and keeps track of how many open/close brackets it has encountered.

But with tree-sitter API, the parser would fill in missing closing brackets but label it as missing. So to find out whether there's unbalanced bracket, and if there is what is its byte offset, you can simply iterate over all child nodes of a syntax tree. And if a node with name }, ], or ) but labelled as missing, then it tells you it's unbalanced.

Similarly, to find out whether a token is enclosed_by_quotes, is just simple locate the token in the tree, using the token's byte offset, and if it's in a node named 'string', it's enclosed in quotes.

detect_comments is simply to find a node named "comment".

Some other example:
To provide semantic highlighting is simply finding all identifier node with same name in a node's parent node or ancestor nodes.
To find all references of a variable is to find all identifier node with same name in its sibling nodes (after it) and its child nodes. (Yes the search should end when the variable is shadowed. But you get the idea here.)

A more severe performance issue is that, the current implementation calculate byte offset by converting strings to UTF-16 BE. Well this is okay by itself, but in the meantime the C program does is to convert a line to a R String (from SEXP) and encode it in UTF-8 if it's not already.

This back-and-forth conversion just looks awful.

Using tree-sitter, you just need to parse the document to get a Tree and don't need to worry about the byte offset, row index, or column index as they are all included in a Node.

When the document is changed, you simply update the end byte offset and end row/column index. And have the parser parse the document again. Note that tree-sitter would reuse nodes of the old tree and only create new nodes when it's necessary.

There's no need to worry about encoding. tree-sitter accepts both UTF-8 and UTF-16 input.

Start by reading src/search.c, especially find_unbalanced_bracket, enclosed_by_quotes, and detect_comments, then review how the current implementation calculates byte offsets. The work is done when tree-sitter-r parsing replaces the manual syntax and encoding logic described in the issue, including incremental document updates.