pygments / pygments/pygments

Wrong processing of \, {, } in C/C++ macros if is used with option `texcomments'

Open
#793 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

S-major T-bug X-imported
Dominant language
Python
Stars
2.2k
Forks
885
PR merge metrics
No merged PRs in 30d

Description

(Original issue 1086 created by LDVSOFT on 2015-02-17T22:28:08.978423+00:00)

While processing C/C++ macros for LaTeX (via minted 2.0) with option texcomments, Pygments process backslash \ and braces {} not as a part of C/C++ language but as a start of TeX command. This fails whole typesetting. For example, I want to pygmentize that:

#define DefineArray(name, type) \
class name{ \
    type *date; \
};

adding texcomments option will turn that output:

\begin{Verbatim}[commandchars=\\\{\}]
\PYGdefault{c+cp}{\PYGdefaultZsh{}define DefineArray(name, type) \PYGdefaultZbs{}}
\PYGdefault{c+cp}{class name\PYGdefaultZob{} \PYGdefaultZbs{}}
\PYGdefault{c+cp}{   type *date; \PYGdefaultZbs{}}
\PYGdefault{c+cp}{\PYGdefaultZcb{};}
\end{Verbatim}

to that:

\begin{Verbatim}[commandchars=\\\{\},codes={\catcode`\$=3\catcode`\^=7\catcode`\_=8}]
\PYGdefault{c+cp}{\PYGdefaultZsh{}define DefineArray(name, type) \}
\PYGdefault{c+cp}{class name{ \}
\PYGdefault{c+cp}{   type *date; \}
\PYGdefault{c+cp}{};}
\end{Verbatim}

that is obviously wrong.

This happens, thanks to developer of minted package gpoore, because preprocessor token in Pygments is Comment.Preproc, and texcomments starts working there. The possible solution could be:

  • Separate preprocessor to new token. Seems logical because preprocessor code is not actually a comment.
  • Add option to ban some Comment.Preproc-alike tokens for texcomments-like options.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the supplied C/C++ macro with the texcomments option, then inspect how the C/C++ lexer emits Comment.Preproc tokens and how minted consumes them for TeX output. Compare the generated output with the expected Verbatim block and make the handling preserve literal backslashes and braces without breaking ordinary TeX comments.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cpp, latex, python
Domain
devtools, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.