jgm / jgm/skylighting

Stata highlighting wrongly highlights keywords contained in comments

Open
#185 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
218
Forks
67
PR merge metrics
No merged PRs in 30d

Description

In Stata's built-in code editor, all text contained in a comment is highlighted as a comment. For example:

![image](https://github.com/jgm/skylighting/assets/47256431/1d28a398-f183-49ea-a15b-030b385bb3df)

In Pandoc outputs, plain text in comments that also happens to be Stata keywords is highlighted as a keyword. For example:

![image](https://github.com/jgm/skylighting/assets/47256431/0c26c549-5cbc-4e8c-8069-a590e23dec38)

Note that [adopath](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L800), [BASE](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L139), [means](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L1375), and [until](https://github.com/jgm/skylighting/blob/bc6401c5dba103b85944c35913ea3ebe9f1decd4/skylighting-core/xml/stata.xml#L266)--all incorrectly highlighted words following comments--appear as words to highlight in stata.xml.

When inspecting the HTML output, one can see the keyword class being applied. Furthermore, the text after comments appears to be "tokenized", and each "token" gets a different highlighting style depending what class it belongs to (e.g., keyword, list of commands, etc.)

![image](https://github.com/jgm/skylighting/assets/47256431/01c3eae3-74f3-4bf1-85d0-e93053d5a4be)

Here's how I produced the HTML output in the last two images above.

1. Create a Markdown file

````
---
title: Hello
---

Here's some Stata code:

```stata
* Set user root folder
global root "C:\Users\user123\github\myproject"

* Set PLUS to adopath and list it first, then list BASE first.
* This means that BASE is first and PLUS is second.
adopath ++ "${root}/code/ado"
adopath ++ BASE

* Keep removing adopaths with rank 3 until only BASE and the project ado-folder,
* that has rank 1 and 2, are left in the adopaths
local morepaths 1
while (`morepaths' == 1) {
capture adopath - 3
if _rc local morepaths 0
}
```

````

2. Render as HTML with Pandoc

```
pandoc stata_test.md -f markdown -t html -s -o stata_test.html
```

Note: I've not (yet) investigated whether this issue also arises for other comments (e.g., single-line comments starting with `//`, end-of-line comments with `///`, or multi-line comments starting with `/*` and ending with `*/`).

Sorry if I'm posting this in the wrong place, or providing less than helpful information.

`skylighting` is a really amazing tool. I'm coming to it from a project that uses [Quarto](https://quarto.org/) to write HTML documentation for Stata packages.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with skylighting-core/xml/stata.xml and reproduce the issue using the Markdown example and the supplied Pandoc command. Inspect the generated HTML to compare comment text with actual Stata code keywords. Done means words such as adopath, BASE, means, and until inside comments no longer receive keyword styling, while real keywords outside comments still do.

Written by the indexing model from the issue text.

Assessment

Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.