fletcher / fletcher/MultiMarkdown-6

Inconsistency? PAIR_HTML_COMMENT token not available in BLOCK_HTML

Open
#221 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
678
Forks
94
PR merge metrics
No merged PRs in 30d

Description

I noticed that when one uses HTML comments inside a regular paragraph, the token tree contains `PAIR_HTML_COMMENT → HTML_COMMENT_START` and eventually `PAIR_HTML_COMMENT → HTML_COMMENT_STOP` (the → denotes is-a-child)

This makes applying highlight colors to HTML comments a bit weird since the comment range is sometimes available as `PAIR_HTML_COMMENT`, sometimes not. Maybe something odd is underlying all this that you might want to know about @fletcher

## Observations

When you put the HTML comment on its own line in an empty document or at the _end_ of a document _without a trailing newline_, it's the same, e.g.:

```
BLOCK_PARA
PAIR_HTML_COMMENT
HTML_COMMENT_START
TEXT_PLAIN ...
HTML_COMMENT_STOP [after this is EOF]
```

But when the HTML comment spans the whole line and ends in a newline character, the token tree is:

```
BLOCK_HTML
HTML_COMMENT_START
TEXT_PLAIN ...
HTML_COMMENT_STOP
TEXT_NL
BLOCK_EMPTY [here is EOF]
```

Note that this seems to be a special case, because the following in a Markdown document ...

```

```

... produces `BLOCK_HTML` with multiple `LINE_HTML` tokens, _except_ for the line with the HTML comment; the tokens from that line stand alone. Here's some debug output with token types and the ranges (location + length):

```
{0,35}TokenType.blockHtml
{0,6}TokenType.lineHtml


{0,1}TokenType.angleLeft
{1,3}TokenType.textPlain
{4,1}TokenType.angleRight
{5,1}TokenType.textNl
{6,4}TokenType.htmlCommentStart
{27,1}TokenType.textNl
{28,7}TokenType.lineHtml

{28,1}TokenType.angleLeft
{29,1}TokenType.slash
{30,3}TokenType.textPlain
{33,1}TokenType.angleRight
{34,1}TokenType.textNl
{35,0}TokenType.blockEmpty
```

### Exceptions from the rule?

Oddly enough, when you have a multi-line HTML comment:

```

```

This is not at all recognized as `BLOCK_HTML`, but `BLOCK_PARA` instead, and it includes a `PAIR_HTML_COMMENT`.

Plus if you add empty newlines around "foo" here, the whole `HTML_COMMENT_PAIR` is dissolved again and you have multiple paragraph blocks, one including the start, one the stop token.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.