Python extractor reports parse errors for files no longer in the scanned commit

Aperta
#22,579 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
devtools

Direzione di ricerca

Inizia riproducendo i diagnostici dell’estrattore Python sul commit 996e624, in cui i file segnalati sono stati eliminati, e confronta l’insieme di file analizzati dall’estrattore con il commit estratto. Usa python -m py_compile come baseline della sintassi. Il lavoro è completato quando i file accettati non producono falsi errori di parsing e i file eliminati non vengono segnalati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

question

Setup

Observed behavior

CodeQL reports parse errors about two Python files. Here's a log sample (I've replaced my actual filenames with foo.py and bar.py):

Analysis produced the following diagnostic information:
##[group]Could not process some files due to syntax errors (2 results)
  * foo.py#L0C0:0: A parse error occurred while processing `foo.py`, and as a result this file could not be analyzed. Check the syntax of the file using the `python -m py_compile` command and correct any invalid syntax.
  * bar.py#L0C0:0: A parse error occurred while processing `bar.py`, and as a result this file could not be analyzed. Check the syntax of the file using the `python -m py_compile` command and correct any invalid syntax.
##[endgroup]

What I tried

  • I ran python -m py_compile on both files. This check completed without errors on Python 3.7, 3.8, 3.9, 3.10, 3.11, and 3.12.

  • I removed an old header from the files: # -*- coding: future_fstrings -*-. This header was a leftover from a Python 2 to Python 3 migration involving https://github.com/asottile-archive/future-fstrings. The warnings stayed. (I imagine, but am not sure, that this header caused the original parsing problem. These two files were the only ones in the repo that still had it.)

  • I checked the files for invisible characters and for a byte order mark. The only non-ASCII characters were some bullet characters inside one docstring. I replaced them with ASCII hyphens. The warnings stayed.

  • I replaced the full contents of both files with a single pass statement. The warnings stayed.

  • I deleted both files. The warnings stayed. I believe these logs confirm that CodeQL scanned the commit where I deleted the files (commit 996e624 in pull request 3584):

    ##[group]Checking out the ref
    [command]/usr/bin/git checkout --progress --force refs/remotes/pull/3584/head
    Note: switching to 'refs/remotes/pull/3584/head'.
    
    You are in 'detached HEAD' state. You can look around, make experimental
    changes and commit them, and you can discard any commits you make in this
    state without impacting any branches by switching back to a branch.
    
    If you want to create a new branch to retain commits you create, you may
    do so (now or later) by using -c with the switch command. Example:
    
      git switch -c <new-branch-name>
    
    Or undo this operation with:
    
      git switch -
    
    Turn off this advice by setting config variable advice.detachedHead to false
    
    HEAD is now at 996e624 bisect: blow away the whole python modules to see if codeql still complains [skip tests]
    ##[endgroup]
    
Expected behavior
  • CodeQL must not report parse errors for Python files that python -m py_compile accepts.
  • CodeQL must not report parse errors for files that do not exist in the scanned commit.
Lingua principale
CodeQL
Stelle
10.1k
Fork
2.1k
Merge medio
2g 11h
PR unite (30g)
129

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di github/codeql

Tutte le issue di github/codeql

Issue simili

Altre issue su DevTools

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.