Update codeql.yml to exclude YAML front-matter and Liquid code
- Dominant language
- JavaScript
- Stars
- 363
- Forks
- 872
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 18
Description
### Overview
Many of our Javascript and HTML code files cannot be scanned by CodeQL as-is because they contain non-JS Liquid code `{% ... %}` or YAML front matter `--- ... ---`, which cause syntax errors. We need to try and resolve these errors without removing all non-JS code.
### Details
The error message "Could not process some files due to syntax errors" indicates that these "syntax errors" may prevent CodeQL from scanning the files below (see issue #5234 for details).
* `hamburger-nav.js`: YAML front-matter with a title
* `toolkit.js`: 1 line of Liquid, empty YAML front-matter
* `wins.js` : 2 lines (Liquid), empty YAML front-matter
* `project.js` : 2 lines (Liquid), empty YAML front-matter
* `about.js`: for loop (Liquid), empty YAML front-matter
* `current-project.js`: 2 lines + for loop (Liquid), empty YAML front-matter
* Separately, we have observed problems with CodeQL scanning of HTML with embedded liquid statements - see #6485
Screenshot: CodeQL error message

Simply deleting the Liquid lines would break the site (and CodeQL raised those errors accordingly in testing), so an alternative, holistic solution is required.
### Action Items
- [x] Review the Possible Solutions content under Resources
- [ ] Implement a solution that will exclude YAML front-matter and Liquid code from CodeQL scans on .js and .html files.
- [ ] Thoroughly test your changes and ensure the codeql.yml file runs as expected. If it does not run as expected, detail your testing in a comment.
### Testing
- [ ] Test your solution by running the `.codeql-scan-job.yml` workflow.
- [ ] You'll have to figure out a way to confirm that CodeQL was able to scan the files listed above without scanning the YAML front-matter or Liquid code and without throwing the error.
### Resources/Instructions
- This issue resulted from #6378
- [Hack for LA's GitHub Actions Wiki](https://github.com/hackforla/website/wiki/Hack-for-LA's-GitHub-Actions)
- https://github.com/hackforla/website/issues/5234#issuecomment-1949927685
#### Possible Solutions
**Here are two possible solutions (in order of preference) to this problem.** Please use your best judgment, these are only recommendations.
#### Option 1
This approach is preferred because it is
- more generic and reusable
- compatible with using a CodeQL for VS Code extension (See [Setting up CodeQL in Visual Studio](https://codeql.github.com/docs/codeql-for-visual-studio-code/setting-up-codeql-in-visual-studio-code/))
Define a new CodeQL query file that excludes Liquid and YAML patterns within JavaScript files.
Create a file named exclude-patterns.ql
```
import javascript
from File file
where (file.getExtension() = "js" or file.getExtension() = "html")
and not file.getCode().matches(".*\\{%.*%\\}.*") // Exclude Liquid code
and not file.getCode().matches(".*---.*") // Exclude YAML front matter
select file
```
Then modify codeql-scan-job.yml file to use the new query file for analysis. Update the queries section in the Initialize CodeQL step to include the new query file:
```
# On codeql-scan-job.yml file:
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
queries: path/to/exclude-patterns.ql, security-and-quality
```
#### Option 2
Exclude liquid code and YAML front matter patterns from the CodeQL analysis within `codeql-scan-job.yml`
```
# On codeql-scan-job.yml file:
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
languages: javascript
queries: security-and-quality
# Exclude Liquid and YAML code within JavaScript files
exclude: |
path: "**/*.{js,html}"
patterns:
- pattern: |
// Start of Liquid code
{% if variable %}
// Liquid code here
{% endif %}
// End of Liquid code
- pattern: |
// Start of YAML front matter
---
# YAML front matter here
---
// End of YAML front matter
```
Contributor guide
Assessment
This issue has not been assessed yet.