DependencyTrack / DependencyTrack/dependency-track

BOM import fails silently when a purl contains non-ASCII characters (packageurl-java 1.5.0 corrupts percent-encoding, overflowing PURLCOORDINATES)

Open
#7,314 0 comments 0 reactions 0 assignees View on GitHub
defect in triage
Dominant language
Java
Stars
4.2k
Forks
811
Avg merge
8h 39m
Merged PRs (30d)
237

Description

### Current Behavior

Uploading a CycloneDX BOM that contains a component whose `purl` includes percent-encoded non-ASCII UTF-8 characters (e.g. `%E3%81%AE` for Japanese "の") causes the **entire BOM import to fail** — no components are ingested.

Worse, the failure is **completely silent from the user's perspective** (the general "no feedback on BOM processing failure" problem is already tracked in #1665, #2890 and #5809 — this issue focuses on the encoding defect that triggers it, but it's worth noting the impact is amplified by that gap):

- The upload API returns a token as usual (HTTP 200)
- The UI shows a successful upload; the project simply stays at 0 components with no error indication
- `/api/v1/bom/token/{token}` flips `processing` to `false` with no error state
- A `BOM_PROCESSING_FAILED` notification is emitted internally, but reaches nobody unless an alert publisher is explicitly configured

The only evidence is in the server log:

```
ERROR [BomUploadProcessingTask] Failed to process BOM [...]
javax.jdo.JDOFatalUserException: Attempt to store value "pkg:nuget/Vsdbg%20%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF81%FFFFFFC2%FFFFFFAE%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF82%FFFFFFC2%FFFFFFAF...
%20vsdbg@18.10.10709.3%20commit%3A9e45c4be6a7fd2ba9b62caae9dec05e66893094d"
in column "PURLCOORDINATES" that has maximum length of 786. Please correct your data!
at org.datanucleus.api.jdo.JDOAdapter.getJDOExceptionForNucleusException(JDOAdapter.java:681)
at org.datanucleus.api.jdo.JDOPersistenceManager.flush(JDOPersistenceManager.java:2057)
at org.dependencytrack.tasks.BomUploadProcessingTask.processComponents(BomUploadProcessingTask.java:476)
...
```

(Same failure on v5.1.0, where the stack trace goes through `org.dependencytrack.tasks.ImportBomActivity.processBom` and the PostgreSQL column limit is 1024.)

### Root cause

> Note: the following root cause analysis was performed with AI assistance (Fable 5) and verified manually against the packageurl-java 1.5.0 sources and a local Dependency-Track deployment.

The purl in the BOM is only **286 characters** and perfectly valid. It gets inflated ~4.3x by two bugs in `packageurl-java` **1.5.0** (the version pinned by both DT 4.14.4 and DT 5.1.0):

1. **`uriDecode` decodes UTF-8 multi-byte sequences byte-by-byte**
([PackageURL.java#L492-L510](https://github.com/package-url/packageurl-java/blob/packageurl-java-1.5.0/src/main/java/com/github/packageurl/PackageURL.java#L492-L510)):
`%E3%81%AE` (Japanese "の", one 3-byte UTF-8 character) is decoded into three separate chars `\u00E3` + `\u0081` + `\u00AE`, which re-encode to **6** UTF-8 bytes (`C3 A3 C2 81 C2 AE`) instead of 3.

2. **`uriEncode` sign-extends bytes ≥ 0x80**
([PackageURL.java#L443-L457](https://github.com/package-url/packageurl-java/blob/packageurl-java-1.5.0/src/main/java/com/github/packageurl/PackageURL.java#L443-L457)):

```java
builder.append('%');
builder.append(Integer.toHexString(b).toUpperCase()); // byte 0xC3 -> int -61 -> "FFFFFFC3"
```

Every non-ASCII byte becomes `%FFFFFFC3` (9 chars) instead of `%C3` (3 chars).

Net effect: each non-ASCII character in the original purl expands from 9 chars (`%E3%81%AE`) to 54 chars (`%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF81%FFFFFFC2%FFFFFFAE`) when DT canonicalizes the purl to populate `PURLCOORDINATES`, blowing past the column limit. DataNucleus then aborts the whole import transaction.

The encoding rewrite on packageurl-java `master` (upcoming 2.0.0, [StringUtil.percentEncode](https://github.com/package-url/packageurl-java/blob/master/src/main/java/com/github/packageurl/internal/StringUtil.java)) fixes both bugs, but 2.0.0 is not yet released on Maven Central (latest is 1.5.0 from Dec 2023).

Real-world trigger: syft scanning a machine with `vsdbg` — the Microsoft .NET debugger that VS Code's C# extension (and Visual Studio remote debugging) installs **automatically** on any machine where .NET debugging is used, so it is present on a very large number of developer machines, build agents, and remote/embedded targets. Its localized resource DLLs (`~/.vs-debugger/install/{cs,de,es,fr,it,ja,ko,pl,pt-BR,ru,tr,...}/VsDbg.SymbolLocator.resources.dll`) produce nuget purls whose names are localized descriptions in Turkish/Japanese/Russian/etc. One such component in a full-system SBOM kills the entire import.

### Steps to Reproduce

1. Start Dependency-Track:

```yaml
# docker-compose.yml
services:
dtrack-apiserver:
image: dependencytrack/apiserver:4.14.4
environment:
- ALPINE_DATABASE_MODE=embedded
ports:
- "8081:8080"
volumes:
- dtrack-data:/data
volumes:
dtrack-data:
```

2. Save this minimal BOM as `minimal-sbom.json` (single component, real-world purl produced by syft from `vsdbg`'s Japanese resource DLL — 286 chars, well within limits):

```json
{
"bomFormat": "CycloneDX",
"specVersion": "1.7",
"version": 1,
"components": [
{
"type": "library",
"bom-ref": "repro-component",
"name": "Vsdbg のクロスプラットフォーム .NET シンボル ロケーター",
"version": "18.10.10709.3 commit:9e45c4be6a7fd2ba9b62caae9dec05e66893094d",
"purl": "pkg:nuget/Vsdbg%20%E3%81%AE%E3%82%AF%E3%83%AD%E3%82%B9%E3%83%97%E3%83%A9%E3%83%83%E3%83%88%E3%83%95%E3%82%A9%E3%83%BC%E3%83%A0%20.NET%20%E3%82%B7%E3%83%B3%E3%83%9C%E3%83%AB%20%E3%83%AD%E3%82%B1%E3%83%BC%E3%82%BF%E3%83%BC@18.10.10709.3%20commit%3A9e45c4be6a7fd2ba9b62caae9dec05e66893094d"
}
]
}
```

3. Upload it (via UI, or API):

```bash
API=http://localhost:8081
curl -s -X POST "$API/api/v1/user/forceChangePassword" \
--data-urlencode "username=admin" --data-urlencode "password=admin" \
--data-urlencode "newPassword=Repro123!" --data-urlencode "confirmPassword=Repro123!"
JWT=$(curl -s -X POST "$API/api/v1/user/login" \
--data-urlencode "username=admin" --data-urlencode "password=Repro123!")
curl -s -X PUT "$API/api/v1/bom" \
-H "Authorization: Bearer $JWT" -H 'Content-Type: application/json' \
-d "{\"projectName\":\"purl-repro\",\"projectVersion\":\"1.0\",\"autoCreate\":true,\"bom\":\"$(base64 -w0 minimal-sbom.json)\"}"
```

4. Observe:
- Upload returns HTTP 200 with a token
- Project `purl-repro` exists but has 0 components; no error visible in UI or API
- `docker compose logs dtrack-apiserver` contains the `JDOFatalUserException` / `PURLCOORDINATES` error above

### Alternative: full end-to-end reproduction from scratch

Instead of step 2, generate the BOM the same way it occurs in the wild — download `vsdbg` using Microsoft's official install script (the same one VS Code runs) and scan it with [syft](https://github.com/anchore/syft):

```bash
# download vsdbg (installs the localized resource DLLs)
curl -sSL https://aka.ms/getvsdbgsh | bash /dev/stdin -v latest -l ./vsdbg-install

# generate a CycloneDX SBOM from it
syft dir:./vsdbg-install -o cyclonedx-json=minimal-sbom.json

# show the offending purls (one per localization)
jq -r '.components[] | select(.purl != null and (.purl | test("%[89A-F][0-9A-F]"))) | .purl' minimal-sbom.json
```

Uploading that BOM in step 3 fails identically.

### Expected Behavior

- A valid purl with percent-encoded UTF-8 characters should be canonicalized correctly (per the purl spec) and stored without inflation — i.e. don't corrupt `%E3%81%AE` into `%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF81%FFFFFFC2%FFFFFFAE`.
- A single malformed/oversized component should not abort the entire BOM import — skip it with a warning, or fail visibly (feedback gap already tracked in #1665 / #2890).

## Workarounds we found

- Exclude the vsdbg directory from syft scans: `--exclude '**/.vs-debugger/**'`
- Or strip affected components from an existing BOM: `jq` filter removing components whose purl matches `%[89A-F][0-9A-F]` percent-encodings
- Configure a `BOM_PROCESSING_FAILED` alert publisher to at least make the failure visible

### Dependency-Track Version

4.x

### Browser

Microsoft Edge

### Checklist

- [x] I have read and understand the [contributing guidelines](https://github.com/DependencyTrack/dependency-track/blob/main/CONTRIBUTING.md#filing-issues)
- [x] I have checked the [existing issues](https://github.com/DependencyTrack/dependency-track/issues) for whether this defect was already reported

Contributor guide

Open the contributing guide

Research direction

Start at org.dependencytrack.tasks.BomUploadProcessingTask.processComponents, and compare the v5.1.0 path through ImportBomActivity.processBom. Reproduce the failure with minimal-sbom.json and inspect the pinned packageurl-java 1.5.0 behavior. Done means percent-encoded UTF-8 purls are stored without inflation and the BOM import no longer aborts on this component.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, postgresql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.