Instagram / Instagram/LibCST

Associate comment of first statement properly in a Module

Open
#405 4 comments 0 reactions 0 assignees View on GitHub
parsing
Dominant language
Python
Stars
1.9k
Forks
229
PR merge metrics
No merged PRs in 30d

Description

In a local scope, e.g. inside a function, class, if condition.
The comment is associated with the statement correctly.
```
In [10]: code = """
...: def f():
...: # abc
...: import d
...: """

In [11]: cst.parse_module(code)
Out[11]:
Module(
body=[
FunctionDef(
name=Name(
value='f',
lpar=[],
rpar=[],
),
...
body=IndentedBlock(
body=[
SimpleStatementLine(
body=[
Import(
names=[
ImportAlias(
name=Name(
value='d',
lpar=[],
rpar=[],
),
asname=None,
comma=MaybeSentinel.DEFAULT,
),
],
semicolon=MaybeSentinel.DEFAULT,
whitespace_after_import=SimpleWhitespace(
value=' ',
),
),
],
leading_lines=[
EmptyLine(
indent=True,
whitespace=SimpleWhitespace(
value='',
),
comment=Comment(
value='# abc',
),
newline=Newline(
value=None,
),
),
],
trailing_whitespace=TrailingWhitespace(
whitespace=SimpleWhitespace(
value='',
),
comment=None,
newline=Newline(
value=None,
),
),
),
],
...
)
```

However, in a module, the comment is parsed as module headers. So some codemods may insert statement between the comment and the first statement, e.g. `EnsureImportPresentCommand `.
```
In [8]: code = """
...: # abc
...: import d
...: """

In [9]: cst.parse_module(code)
Out[9]:
Module(
body=[
SimpleStatementLine(
body=[
Import(
names=[
ImportAlias(
name=Name(
value='d',
lpar=[],
rpar=[],
),
asname=None,
comma=MaybeSentinel.DEFAULT,
),
],
semicolon=MaybeSentinel.DEFAULT,
whitespace_after_import=SimpleWhitespace(
value=' ',
),
),
],
leading_lines=[],
trailing_whitespace=TrailingWhitespace(
whitespace=SimpleWhitespace(
value='',
),
comment=None,
newline=Newline(
value=None,
),
),
),
],
header=[
EmptyLine(
indent=True,
whitespace=SimpleWhitespace(
value='',
),
comment=None,
newline=Newline(
value=None,
),
),
EmptyLine(
indent=True,
whitespace=SimpleWhitespace(
value='',
),
comment=Comment(
value='# abc',
),
newline=Newline(
value=None,
),
),
],
footer=[],
encoding='utf-8',
default_indent=' ',
default_newline='\n',
has_trailing_newline=True,
)
```

It's probably not easy to differentiate whether a comment is for the first statement or not to handle it properly when parsing a module. It could be solved by building a helper to post processing the CST by giving some hints (e.g. `# lint-ignore` comment should be associate with the first statement).

CC @zsol @thatch

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.