tree-sitter / tree-sitter/tree-sitter-cpp

bug: Parse error in string concatenation with macros

Open
#307 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
JavaScript
Stars
453
Forks
185
PR merge metrics
No merged PRs in 30d

Description

Did you check existing issues?
  • I have read all the tree-sitter docs if it relates to using the parser
  • I have searched the existing issues of tree-sitter-cpp
Tree-Sitter CLI Version, if relevant (output of tree-sitter --version)

tree-sitter 0.25.3 (2a835ee029dca1c325e6f1c01dbce40396f6123e)

Describe the bug

The parser does not correctly recognize string concatenation when using macros. For example, in the following code:

#define HELLO "Hello"
#define WORLD "World"

int main() {
  const char* s1 = HELLO " World";
  const char* s2 = "Hello " WORLD;
  
  return 0;
}

while the parser correctly recognizes the assignment to s1 (i.e., the concatenation of HELLO and " World") as a concatenated_string, it fails to do the same for the assignment to s2. It does not recognize the concatenation of "Hello " and WORLD as a single concatenated_string, and instead reports and error.

Steps To Reproduce/Bad Parse Tree
  1. Write the code above in a C++ file.
  2. Run tree-sitter parse on the file.
  3. Observe the obtained parse tree:
(translation_unit [0, 0] - [7, 0]
  (preproc_def [0, 0] - [1, 0]
    name: (identifier [0, 8] - [0, 13])
    value: (preproc_arg [0, 14] - [0, 21]))
  (preproc_def [1, 0] - [2, 0]
    name: (identifier [1, 8] - [1, 13])
    value: (preproc_arg [1, 14] - [1, 21]))
  (function_definition [3, 0] - [6, 1]
    type: (primitive_type [3, 0] - [3, 3])
    declarator: (function_declarator [3, 4] - [3, 10]
      declarator: (identifier [3, 4] - [3, 8])
      parameters: (parameter_list [3, 8] - [3, 10]))
    body: (compound_statement [3, 11] - [6, 1]
      (declaration [4, 2] - [4, 34]
        (type_qualifier [4, 2] - [4, 7])
        type: (primitive_type [4, 8] - [4, 12])
        declarator: (init_declarator [4, 12] - [4, 33]
          declarator: (pointer_declarator [4, 12] - [4, 16]
            declarator: (identifier [4, 14] - [4, 16]))
          value: (concatenated_string [4, 19] - [4, 33]
            (identifier [4, 19] - [4, 24])
            (string_literal [4, 25] - [4, 33]
              (string_content [4, 26] - [4, 32])))))
      (declaration [5, 2] - [5, 34]
        (type_qualifier [5, 2] - [5, 7])
        type: (primitive_type [5, 8] - [5, 12])
        declarator: (init_declarator [5, 12] - [5, 27]
          declarator: (pointer_declarator [5, 12] - [5, 16]
            declarator: (identifier [5, 14] - [5, 16]))
          value: (string_literal [5, 19] - [5, 27]
            (string_content [5, 20] - [5, 26])))
        (ERROR [5, 28] - [5, 33]
          (identifier [5, 28] - [5, 33]))))))
Expected Behavior/Parse Tree

The parser should correctly identify both s1 and s2 as concatenated_string, recognizing the concatenation of "Hello " and WORLD in s2 in the same way it handles the concatenation of HELLO and " World" in s1.
As a result, in the parse tree, no error should be detected for s2; instead, it shows an error (ERROR [5, 28] - [5, 33]).

Repro
#define HELLO "Hello"
#define WORLD "World"

int main() {
  const char* s1 = HELLO " World";
  const char* s2 = "Hello " WORLD;
  
  return 0;
}

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running tree-sitter parse from the reported reproduction against tree-sitter-cpp and compare the two assignments' parse trees. Trace the grammar handling concatenated_string and macro identifiers; done means both s1 and s2 parse as concatenated_string without an ERROR node.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.