Stata highlighting wrongly highlights keywords contained in comments
- Dominant language
- Haskell
- Stars
- 218
- Forks
- 67
- PR merge metrics
- No merged PRs in 30d
Description
In Stata's built-in code editor, all text contained in a comment is highlighted as a comment. For example:

In Pandoc outputs, plain text in comments that also happens to be Stata keywords is highlighted as a keyword. For example:

Note that [adopath](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L800), [BASE](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L139), [means](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L1375), and [until](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L266)--all incorrectly highlighted words following comments--appear as words to highlight in stata.xml.
When inspecting the HTML output, one can see the keyword class being applied. Furthermore, the text after comments appears to be "tokenized", and each "token" gets a different highlighting style depending what class it belongs to (e.g., keyword, list of commands, etc.)

Here's how I produced the HTML output in the last two images above.
1. Create a Markdown file
````
---
title: Hello
---
Here's some Stata code:
```stata
* Set user root folder
global root "C:\Users\user123\github\myproject"
* Set PLUS to adopath and list it first, then list BASE first.
* This means that BASE is first and PLUS is second.
adopath ++ "${root}/code/ado"
adopath ++ BASE
* Keep removing adopaths with rank 3 until only BASE and the project ado-folder,
* that has rank 1 and 2, are left in the adopaths
local morepaths 1
while (`morepaths' == 1) {
capture adopath - 3
if _rc local morepaths 0
}
```
````
2. Render as HTML with Pandoc
```
pandoc stata_test.md -f markdown -t html -s -o stata_test.html
```
Note: I've not (yet) investigated whether this issue also arises for other comments (e.g., single-line comments starting with `//`, end-of-line comments with `///`, or multi-line comments starting with `/*` and ending with `*/`).
Sorry if I'm posting this in the wrong place, or providing less than helpful information.
`skylighting` is a really amazing tool. I'm coming to it from a project that uses [Quarto](https://quarto.org/) to write HTML documentation for Stata packages.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with skylighting-core/xml/stata.xml and reproduce the issue using the Markdown example and the supplied Pandoc command. Inspect the generated HTML to compare comment text with actual Stata code keywords. Done means words such as adopath, BASE, means, and until inside comments no longer receive keyword styling, while real keywords outside comments still do.
Written by the indexing model from the issue text.
Assessment
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100