Python extractor reports parse errors for files no longer in the scanned commit
- Ngôn ngữ chính
- CodeQL
- Star
- 10.1k
- Fork
- 2.1k
- Merge trung bình
- 2 ngày 15 giờ
- Pull request đã merge (30 ngày)
- 141
Mô tả
## Setup
- CodeQL version: 2.27.0
- Language: Python 3.13
- GitHub default CodeQL analysis (Advanced Security > Code scanning > Default setup)
- Repo (private): https://github.com/lumin-ai/lumin-bot
## Observed behavior
CodeQL reports parse errors about two Python files. Here's a log sample (I've replaced my actual filenames with `foo.py` and `bar.py`):
Analysis produced the following diagnostic information:
##[group]Could not process some files due to syntax errors (2 results)
* foo.py#L0C0:0: A parse error occurred while processing `foo.py`, and as a result this file could not be analyzed. Check the syntax of the file using the `python -m py_compile` command and correct any invalid syntax.
* bar.py#L0C0:0: A parse error occurred while processing `bar.py`, and as a result this file could not be analyzed. Check the syntax of the file using the `python -m py_compile` command and correct any invalid syntax.
##[endgroup]
## What I tried
- I ran `python -m py_compile` on both files. This check completed without errors on Python 3.7, 3.8, 3.9, 3.10, 3.11, and 3.12.
- I removed an old header from the files: `# -*- coding: future_fstrings -*-`. This header was a leftover from a Python 2 to Python 3 migration involving https://github.com/asottile-archive/future-fstrings. The warnings stayed. (I imagine, but am not sure, that this header caused the original parsing problem. These two files were the only ones in the repo that still had it.)
- I checked the files for invisible characters and for a byte order mark. The only non-ASCII characters were some bullet characters inside one docstring. I replaced them with ASCII hyphens. The warnings stayed.
- I replaced the full contents of both files with a single `pass` statement. The warnings stayed.
- *I deleted both files*. The warnings stayed. I believe these logs confirm that CodeQL scanned the commit where I deleted the files (commit 996e624 in pull request 3584):
```
##[group]Checking out the ref
[command]/usr/bin/git checkout --progress --force refs/remotes/pull/3584/head
Note: switching to 'refs/remotes/pull/3584/head'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 996e624 bisect: blow away the whole python modules to see if codeql still complains [skip tests]
##[endgroup]
```
### Expected behavior
- CodeQL must not report parse errors for Python files that `python -m py_compile` accepts.
- CodeQL must not report parse errors for files that do not exist in the scanned commit.
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu bằng cách tái hiện các chẩn đoán của trình trích xuất Python trên commit 996e624, nơi các tệp được báo cáo đã bị xóa, và so sánh tập tệp được trình trích xuất quét với commit đã checkout. Sử dụng `python -m py_compile` làm cơ sở chuẩn về cú pháp. Được xem là hoàn tất khi các tệp được chấp nhận không tạo ra lỗi phân tích cú pháp giả và các tệp đã xóa không bị báo cáo.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- python
- Lĩnh vực
- devtools
- Loại issue
- Lỗi
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức độ hoạt động
- Sôi nổi
- Độ rõ ràng
- Khá rõ ràng
- Mức phù hợp với người mới
- 48/100