Some feedback on the Org parsing

Hello,

This is a bit of a drive-by issue as I'm not really a neovim user but I thought I'd check out the tree-sitter parsing since I think it looks cool. I've only had a little look, but a few rough spots have become apparent in the three files I checked out.

System setup

• Linux, OpenSUSE Tumbleweed
• neovim version v0.5.1
• orgmode.nvim commit c30b645315e37e77708a0bdfaaa9f05b28530099
• tree sitter parser commit https://github.com/milisims/tree-sitter-org/commit/39a377f5072ee9f79884e227dc49d42c2eba67d8
• nvim config http://ix.io/3FM3 (basically bach's defaults.nvim + recommended setup from readme + conceal)

config.org

This is the first file I looked at, it's my Emacs config file, and it's massive.

Experience

File took a few seconds to open, then I got hit with

Error detected while processing CursorHold Autocommands for "*.org":
E5108: Error executing lua ...ack/packer/start/orgmode.nvim/lua/orgmode/utils/init.lua:297: too many results to unpack

The header keywords are all grey and underlined, and the rest of the document is red and underlined.

Thesis.org (file available on request)

This is the second Org file I tried, it's a ~60 page Thesis I wrote this year. A fair bit is going on, but it's smaller and less complicated than config.org.

Experience

Upon opening the file I immediately see the warning gutentags: ctags job failed, returned: 1.

While there's no error being blasted in my face like with config.org, unfortunately, the results don't look great here either.
All of the header keywords are underlined grey, except for the commented out subtitle which is red for some reason.
The first header, * LaTeX setup :ignore: is also red for some reason.
A few lines later we have the first source code block, and from then on the rest of the document is underlined red except for headings.

2021-04-26-Welcome.org

This is the first file that wasn't mostly underlined red text! 🎉 It's a fairly short and simple file, the first blog post I published.

Experience

Links

I notice that concel can cause some funny visual results with links

For reference, this is how that looks in Emacs,

Quite a few links also seem to be incorrectly parsed, such as

I tried changing the link target to see what was happening and this threw another error

Error detected while processing TextChangedI Autocommands for "*"
E5108: Error while executing lua ....e/pack/packer/start/cmp_luasnip/lua/cmp_luasnip/init.lua:65: attempted to call field 'get_snippet_filetypes' (a nil value)

Ignoring this and trying a few different link targets, it becomes apparent that this issue occurs when the link target spreads over multiple lines.

Inline source blocks

These don't seem to be handled well at the moment.

Meanwhile in Emacs,

It's worth noting that even if src_*{} constructs weren't special, this still shouldn't be handled as an underline. Say I replace src with hey in the Org file. This is what I then see:

Final comments

There are a lot of elements that aren't parsed at all, but I thought it would be best to ignore those for now while there prominent issues with the current parsing.

Lastly, apologies if this isn't as helpful as it could be. As mentioned at the start, I am not at all familiar with neovim (or vim for that matter) and so have just tried to highlight the roughest spots I've noticed. All in all I think it's fantastic to see a genuine effort being made with using tree-sitter based parsing for Org and wish you all the best going forwards. If there's anything I can do to help with a next to no neovim knowledge but a good knowledge of Org and some big/complicated files I'd be happy to assist where I can 🙂.

Start by reproducing the reported parsing and highlighting problems with config.org, Thesis.org, and 2021-04-26-Welcome.org, then inspect lua/orgmode/utils/init.lua:297 where the unpacking error is reported. Compare handling of multiline links, inline source blocks, and large files; done means the reported errors and incorrect highlighting no longer occur.