Python extractor reports parse errors for files no longer in the scanned commit
还没有人认领这个 Issue。
评估
调研方向
首先,针对 commit 996e624 重现 Python 提取器的诊断结果;在该 commit 中,报告的文件已被删除,然后将提取器扫描的文件集与已检出的 commit 进行比较。使用 python -m py_compile 作为语法基线。完成的标准是:已接受的文件不会产生错误的解析错误,已删除的文件不会被报告。
由索引模型根据 Issue 内容生成。
描述
Setup
- CodeQL version: 2.27.0
- Language: Python 3.13
- GitHub default CodeQL analysis (Advanced Security > Code scanning > Default setup)
- Repo (private): https://github.com/lumin-ai/lumin-bot
Observed behavior
CodeQL reports parse errors about two Python files. Here's a log sample (I've replaced my actual filenames with foo.py and bar.py):
Analysis produced the following diagnostic information:
##[group]Could not process some files due to syntax errors (2 results)
* foo.py#L0C0:0: A parse error occurred while processing `foo.py`, and as a result this file could not be analyzed. Check the syntax of the file using the `python -m py_compile` command and correct any invalid syntax.
* bar.py#L0C0:0: A parse error occurred while processing `bar.py`, and as a result this file could not be analyzed. Check the syntax of the file using the `python -m py_compile` command and correct any invalid syntax.
##[endgroup]
What I tried
-
I ran
python -m py_compileon both files. This check completed without errors on Python 3.7, 3.8, 3.9, 3.10, 3.11, and 3.12. -
I removed an old header from the files:
# -*- coding: future_fstrings -*-. This header was a leftover from a Python 2 to Python 3 migration involving https://github.com/asottile-archive/future-fstrings. The warnings stayed. (I imagine, but am not sure, that this header caused the original parsing problem. These two files were the only ones in the repo that still had it.) -
I checked the files for invisible characters and for a byte order mark. The only non-ASCII characters were some bullet characters inside one docstring. I replaced them with ASCII hyphens. The warnings stayed.
-
I replaced the full contents of both files with a single
passstatement. The warnings stayed. -
I deleted both files. The warnings stayed. I believe these logs confirm that CodeQL scanned the commit where I deleted the files (commit 996e624 in pull request 3584):
##[group]Checking out the ref [command]/usr/bin/git checkout --progress --force refs/remotes/pull/3584/head Note: switching to 'refs/remotes/pull/3584/head'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 996e624 bisect: blow away the whole python modules to see if codeql still complains [skip tests] ##[endgroup]
Expected behavior
- CodeQL must not report parse errors for Python files that
python -m py_compileaccepts. - CodeQL must not report parse errors for files that do not exist in the scanned commit.
- 主要语言
- CodeQL
- 星标
- 10.1k
- 派生
- 2.1k
- 平均合并
- 2 天 11 小时
- 30 天内合并 PR
- 129
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
github/codeql 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 84/100
-
难度 2/5 1-3 小时 新手友好度 82/100
-
难度 2/5 1-3 小时 新手友好度 78/100
-
false-positive
难度 2/5 1-3 小时 新手友好度 70/100
-
False positive 未关闭false-positive
难度 4/5 3-5 天 新手友好度 15/100
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 78/100
idean3885/claude-ops-agent#521 ·
-
难度 2/5 1-3 小时 新手友好度 72/100
0xMiden/bridge-portal#132 ·
-
bug
难度 2/5 1-3 小时 新手友好度 84/100
newrelic-experimental/preflight#793 · 1 条评论 ·
-
enhancement
难度 2/5 1-3 小时 新手友好度 70/100
babalae/bettergi-scripts-list#3674 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
caddyserver/caddy#8046 ·