Optimizing the R code parsing

Following the discussion created by @kobytg17 in #599 (brief: we have enormous amount of code in R at our company), I've been reviewing the parsing process and why it's so slow. The main parsing function of this language server is: https://github.com/REditorSupport/languageserver/blob/1e71561a41ef59126273f16bbb466c0b30a2aa64/R/document.R#L402-L429
It seems that the main parsing process, per each file, is done in the following stages:

1. Parse the file by the built-in parse() function of R.
2. Scan the result of 1 (recursive walk on the AST), and save this result in some recursive R Environment object.
3. Encode the result of 1 to a string representing an XML (line 426 in the code above), and then this XML string is being decoded later in: https://github.com/REditorSupport/languageserver/blob/1e71561a41ef59126273f16bbb466c0b30a2aa64/R/workspace.R#L243-L249

I'm trying to see how can we make this parsing process faster (and hopefully takes less amount of RAM).

One option is to handle this problem like RStudio - have the whole language server written only in C++. They don't have a language server, but their core is written only in C++. For getting the impression, see their tokenizing code here: https://github.com/rstudio/rstudio/blob/02e3810fabbca032fcb664196a1c008e6306ac7f/src/cpp/core/include/core/r_util/RTokenizer.hpp#L346. Of course this doesn't seem to me as a reasonable option because of the huge responsibility and the refactoring.

Therefore, I'm focusing on finding ways to optimize the parsing. I see two immediate solutions:

1. Getting rid of step 3 above, since the XML encoding and decoding intuitively seems to be a waste of computation, we must have another way to get the needed parsing features directly.
2. Implement step 2 (recursively walk on the AST and producing a recursive R Environment object ) in raw C++. Later, if the memory footprints are still high(probably it will be since the result will be the same), we can also change the Environment object to a raw C++ recursive struct or something, and implement some more utility AST functions in C++.

What do you think?

Start with R/document.R lines 402-429 and R/workspace.R lines 243-249 to trace parsing, AST walking, XML encoding, and decoding. Compare the proposed R and C++ approaches, then establish benchmarks for parsing time and memory; the issue provides no concrete completion criterion beyond improving both.