github / github/codeql

JavaScript extractor ignores html files although docs (and code) say it may also be analyzed

Open
#21,048 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
CodeQL
Stars
10.1k
Forks
2.1k
Avg merge
2d 15h
Merged PRs (30d)
141

Description

Given the fact that I specify the languages to scan manually, for specific reasons.
Since 2.23.8 finalizing the javascript database now fails because it did not find any suitable file to extract from.

If I take a look at the documentation available at https://codeql.github.com/docs/codeql-overview/supported-languages-and-frameworks/#id9 I see that for the language JavaScript it should also be able to analyze html files (they might contain JavaScript). Given that I should be able to perform code scanning on javascript based on the presence of html files in my sources.

Because in version 2.23.8 the implementation has been changed, it now fails to finalize the javascript database. https://github.com/github/codeql/blob/c4d23d16edfde2566c4986e4dd7b7a177940886d/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java#L493

In the autobuild for JavaScript I do see a bunch of filetypes being added, but they are not detected in the extract phase. https://github.com/github/codeql/blob/c4d23d16edfde2566c4986e4dd7b7a177940886d/javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java#L388

In the fileextractor for JavaScript I do see a bunch of extensions being added for HTML. Which is misleading.
https://github.com/github/codeql/blob/d709343d38f921d5beccf530dc0a0b30a6680a12/javascript/extractor/src/com/semmle/js/extractor/FileExtractor.java#L106

Can you please clarify this?

note:
In the default setup of code scanning the detected languages on the repository are used. If you only happen to have html files in your repository it will not "detect" the JavaScript language, code scanning will simply ignore them.

Contributor guide

Open the contributing guide

Research direction

Start by comparing the JavaScript extractor paths in javascript/extractor/src/com/semmle/js/extractor/AutoBuild.java and FileExtractor.java, using the linked lines and the supported-languages documentation. Reproduce the manual JavaScript scan with HTML-only sources and verify whether extraction sees the HTML files and database finalization succeeds as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, javascript
Domain
devtools, security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.