nikic / nikic/PHP-Parser

Inline comments with # getting associated with the wrong source-code element?

Open
#201 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
17.5k
Forks
1.1k
PR merge metrics
No merged PRs in 30d

Description

I seem to be running into a parser issue, but can't be sure if this is a bug or as-intended. It seems buggy.

Code:

$code = '<?php

$foo = 1;

$foo !== 2; # just because

$bar = 2;
';

$nodes = $parser->parse($code);

var_dump($nodes);

Output:

array(3) {
  [0] =>
  class PhpParser\Node\Expr\Assign#10 (4) {
    public $var =>
    class PhpParser\Node\Expr\Variable#8 (3) {
      public $name =>
      string(3) "foo"
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(3)
        'endLine' =>
        int(3)
      }
    }
    public $expr =>
    class PhpParser\Node\Scalar\LNumber#9 (3) {
      public $value =>
      int(1)
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(3)
        'endLine' =>
        int(3)
      }
    }
    private $subNodeNames =>
    NULL
    protected $attributes =>
    array(2) {
      'startLine' =>
      int(3)
      'endLine' =>
      int(3)
    }
  }
  [1] =>
  class PhpParser\Node\Expr\BinaryOp\NotIdentical#13 (4) {
    public $left =>
    class PhpParser\Node\Expr\Variable#11 (3) {
      public $name =>
      string(3) "foo"
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(5)
        'endLine' =>
        int(5)
      }
    }
    public $right =>
    class PhpParser\Node\Scalar\LNumber#12 (3) {
      public $value =>
      int(2)
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(5)
        'endLine' =>
        int(5)
      }
    }
    private $subNodeNames =>
    NULL
    protected $attributes =>
    array(2) {
      'startLine' =>
      int(5)
      'endLine' =>
      int(5)
    }
  }
  [2] =>
  class PhpParser\Node\Expr\Assign#17 (4) {
    public $var =>
    class PhpParser\Node\Expr\Variable#15 (3) {
      public $name =>
      string(3) "bar"
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(3) {
        'comments' =>
        array(1) {
          [0] =>
          class PhpParser\Comment#14 (2) {
            protected $text =>
            string(16) "# just because\r\n"
            protected $line =>
            int(5)
          }
        }
        'startLine' =>
        int(7)
        'endLine' =>
        int(7)
      }
    }
    public $expr =>
    class PhpParser\Node\Scalar\LNumber#16 (3) {
      public $value =>
      int(2)
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(7)
        'endLine' =>
        int(7)
      }
    }
    private $subNodeNames =>
    NULL
    protected $attributes =>
    array(3) {
      'comments' =>
      array(1) {
        [0] =>
        class PhpParser\Comment#14 (2) {
          protected $text =>
          string(16) "# just because\r\n"
          protected $line =>
          int(5)
        }
      }
      'startLine' =>
      int(7)
      'endLine' =>
      int(7)
    }
  }
}

The comment trailing the $foo !== 2; expression (index 1) somehow gets associated with the $bar = 2; assignment (index 2) which seems odd.

Maybe the intention here is for doc-blocks to get correctly associated with the node following the doc-block... Are all comments deliberately associated with the following node whether they're doc-blocks or not?

That seems odd, and appears to be lead to orphaned last comment nodes - example:

$code = '<?php

$foo !== 2; # just because
';

$nodes = $parser->parse($code);

var_dump($nodes);

Output:

array(1) {
  [0] =>
  class PhpParser\Node\Expr\BinaryOp\NotIdentical#10 (4) {
    public $left =>
    class PhpParser\Node\Expr\Variable#8 (3) {
      public $name =>
      string(3) "foo"
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(3)
        'endLine' =>
        int(3)
      }
    }
    public $right =>
    class PhpParser\Node\Scalar\LNumber#9 (3) {
      public $value =>
      int(2)
      private $subNodeNames =>
      NULL
      protected $attributes =>
      array(2) {
        'startLine' =>
        int(3)
        'endLine' =>
        int(3)
      }
    }
    private $subNodeNames =>
    NULL
    protected $attributes =>
    array(2) {
      'startLine' =>
      int(3)
      'endLine' =>
      int(3)
    }
  }
}

The comment is lost.

I don't know how you'd get around that, since comments are associated forwards with the next node, if there is no next node. I guess you'd need something like an "end of file" node with which the orphaned trailing comment could be associated?

Even so, that would seem kind of patchy, since what's causing the problem in the first place, is the fact that comments aren't treated as nodes, but as "attributes" of nodes - which is true of /** doc-blocks; these are attributes of the following node - but other types of comments really aren't.

Perhaps this is consistent with the native PHP lexer/parser, and if so, an "end of file" node might be the most reasonable solution (?)

I'm using the release 1.3.0 version.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the two PHP snippets with the parser and inspect how inline comments are attached to AST nodes. Compare the comment placement after $foo !== 2; and the trailing-comment case, then determine and document a consistent expected association or preservation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.