pygments / pygments/pygments

cobol lexer issues

Open
#1,926 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.2k
Forks
885
PR merge metrics
No merged PRs in 30d

Description

Bug in https://github.com/pygments/pygments/blob/master/pygments/lexers/business.py

Given the test program

       IDENTIFICATION DIVISION.
       PROGRAM-ID.             TESTFUNC.
      *AUTHOR.                 ARNOLD J. TREMBLEY.
      *DATE-WRITTEN.           2020-06-30.

       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01  800-WIN-USERNAME                PIC X(24)   VALUE "UNKNOWN".
       01  800-WIN-USERPROFILE             PIC X(24)   VALUE "UNKNOWN".
       01  800-WIN-USERDOMAIN              PIC X(24)   VALUE "UNKNOWN".
       01  800-WHEN-COMPILED.
           05  800-COMPILED-DATE-YYYY      PIC X(04)   VALUE SPACES.
           05  800-COMPILED-DATE-MM        PIC X(02)   VALUE SPACES.
           05  800-COMPILED-DATE-DD        PIC X(02)   VALUE SPACES.
           05  800-COMPILED-TIME-HH        PIC X(02)   VALUE SPACES.
           05  800-COMPILED-TIME-MM        PIC X(02)   VALUE SPACES.
           05  800-COMPILED-TIME-SS        PIC X(02)   VALUE SPACES.
           05  FILLER                      PIC X(07)   VALUE SPACES.
       01  800-CURRENT-DATE.
           05  800-CURRENT-DATE-YYYY       PIC X(04)   VALUE SPACES.
           05  800-CURRENT-DATE-MM         PIC X(02)   VALUE SPACES.
           05  800-CURRENT-DATE-DD         PIC X(02)   VALUE SPACES.
           05  800-CURRENT-TIME-HH         PIC X(02)   VALUE SPACES.
           05  800-CURRENT-TIME-MM         PIC X(02)   VALUE SPACES.
           05  800-CURRENT-TIME-SS         PIC X(02)   VALUE SPACES.
           05  FILLER                      PIC X(07)   VALUE SPACES.

       PROCEDURE DIVISION.
       MAINLINE.
           DISPLAY 'TESTFUNC Start - GnuCOBOL 3.1 RC1 30JUN2020'
           MOVE FUNCTION WHEN-COMPILED TO 800-WHEN-COMPILED
           MOVE FUNCTION CURRENT-DATE  TO 800-CURRENT-DATE
           DISPLAY 'TESTFUNC Compiled = '
               800-compiled-date-yyyy '/'
               800-compiled-date-mm   '/'
               800-compiled-date-dd   space
               800-compiled-time-hh   ':'
               800-compiled-time-mm   ':'
               800-compiled-time-ss
           DISPLAY 'TESTFUNC Executed = '
               800-current-date-yyyy '/'
               800-current-date-mm   '/'
               800-current-date-dd   space
               800-current-time-hh   ':'
               800-current-time-mm   ':'
               800-current-time-ss
           accept omitted
           ACCEPT 800-WIN-USERNAME FROM ENVIRONMENT "USERNAME"
           ACCEPT 800-WIN-USERDOMAIN FROM ENVIRONMENT "USERDOMAIN"
           ACCEPT 800-WIN-USERPROFILE FROM ENVIRONMENT "USERPROFILE"
           DISPLAY "USERNAME    = " 800-WIN-USERNAME
           DISPLAY "USERPROFILE = " 800-WIN-USERPROFILE
           DISPLAY "USERDOMAIN  = " 800-WIN-USERDOMAIN
           DISPLAY 'TESTFUNC Successfully Completed'
           STOP RUN.

all variables starting with a number are partial (only the number) highlighted as keyword, both in their definition 01 800-CURRENT-DATE. and in their use MOVE FUNCTION WHEN-COMPILED TO 800-WHEN-COMPILED.

In general the hyphen is not a separator in COBOL so 123-something will always be a user-defined word.

Here are some additional updates to the reserved word lists as those were currently from OpenCOBOL 1.1 while this has gone a long way to GnuCOBOL 3.2 (I think its best to drop the error-token keywords but instead create a context-sensitive token list): gc32res.txt

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pygments/lexers/business.py and reproduce the COBOL highlighting with the sample program, focusing on identifiers such as 800-CURRENT-DATE and 800-WHEN-COMPILED. Compare the lexer’s reserved-word handling with gc32res.txt. Done means hyphenated words beginning with numbers are treated as user-defined words rather than highlighting only their numeric prefix as a keyword.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.