mandiant / mandiant/capa

tree-sitter: support UTF-16 and UTF-8 encodings

Open
#3,149 1 comment 0 reactions 0 assignees View on GitHub
bug gsoc tree-sitter
Dominant language
Python
Stars
6.2k
Forks
726
Avg merge
11d 11h
Merged PRs (30d)
7

Description

Files fail autodetection or crash because they are UTF-16 LE encoded.

Currently, `capa` assumes UTF-8 in two places:
1. `parser.parse(buf)` in `autodetect.py` and `engine.py` doesn't pass an `encoding` parameter, defaulting to UTF-8. Tree-Sitter treats UTF-16 LE null bytes (`0x00`) as syntax errors, causing autodetection to fail and extracting 0 capabilities.
2. `engine.get_str(node)` calls `.decode("utf-8")` without an error strategy, so obfuscated string literals or non-UTF-8 byte sequences crash `capa` with `UnicodeDecodeError`.

### Proposed Solution

1. **Detect UTF-16 Encodings**:
- Check BOM markers (`\xff\xfe` for UTF-16 LE, `\xfe\xff` for UTF-16 BE, `\xef\xbb\xbf` for UTF-8).
- For BOM-less scripts (e.g. where the BOM was stripped), analyze odd vs. even null-byte (`0x00`) frequency in the initial buffer slice to identify UTF-16 LE/BE.

2. **Pass Encoding to Tree-Sitter**:
- Pass the detected encoding to Tree-Sitter: `parser.parse(buf, encoding=encoding)`.
- Tree-Sitter natively parses UTF-16, and the returned node byte ranges (`node.start_byte`, `node.end_byte`) will match the exact physical byte offsets in the raw file on disk.

3. **Safe String Extraction**:
- Update string node decoding in `engine.py` to use the file's native encoding with `errors="replace"`:
```python
def get_str(self, node: Node) -> str:
return self.get_byte_range(node).decode(self.encoding, errors="replace")
```
- This prevents crashes when extracting strings containing non-UTF-8 bytes or raw payload data.

Contributor guide

Open the contributing guide

Research direction

Start by reading autodetect.py and engine.py, focusing on parser.parse(buf), engine.get_str(node), and how the file encoding is currently represented. Run the existing parsing and capability-extraction tests, then verify UTF-8, UTF-16 LE/BE with and without BOMs, and invalid byte sequences. Done means Tree-Sitter receives the detected encoding, byte ranges remain usable, and string extraction no longer crashes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
reverse-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.