github / github/codeql

Add new state: Unicode compatibility normalization

Open
#19,706 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
CodeQL
Stars
10.1k
Forks
2.1k
Avg merge
2d 15h
Merged PRs (30d)
141

Description

Hey ,

I noticed that you are considering only two states:
1. One regarding the path normalization if it is done or not before the safe check
2. Second concerns the safe check.

as shown next:

https://github.com/github/codeql/blob/c1c0a705b9f14c0f577a9ae56a9d699e8b6e67d6/python/ql/lib/semmle/python/security/dataflow/PathInjectionQuery.qll#L20-L28

However, there is a third state that is a required one: `Unicode normalized`. If ever a Unicode normalization is performed with a compatibility algorithm (NFKC or NFKD), the query would miss some cases precisely those ones where the Unicode normalization is not performed before the path normalization and the safe check. I draw a little chart to depict my saying:

Image

The previous chart shows that when you consider a potential Unicode compatibility normalization, it is a required step before path normalization and safe check. If ever placed between the first two steps or after the last one, that would yield a vulnerable case that got missed due to the fact that the Unicode normalization may reintroduce unexpected special characters such as `..` and `/`.

Regards
@Sim4n6

Contributor guide

Open the contributing guide

Research direction

Start in python/ql/lib/semmle/python/security/dataflow/PathInjectionQuery.qll, especially lines 20-28, and compare the existing path-normalization and safe-check states with the reported Unicode compatibility-normalization state. Use the issue’s chart to trace the required ordering; done means the query models compatibility normalization in the relevant position without missing cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.