DependencyTrack / DependencyTrack/dependency-track
BOM import fails silently when a purl contains non-ASCII characters (packageurl-java 1.5.0 corrupts percent-encoding, overflowing PURLCOORDINATES)
- Dominant language
- Java
- Stars
- 4.2k
- Forks
- 811
- Avg merge
- 8h 39m
- Merged PRs (30d)
- 237
Description
### Current Behavior
Uploading a CycloneDX BOM that contains a component whose `purl` includes percent-encoded non-ASCII UTF-8 characters (e.g. `%E3%81%AE` for Japanese "の") causes the **entire BOM import to fail** — no components are ingested.
Worse, the failure is **completely silent from the user's perspective** (the general "no feedback on BOM processing failure" problem is already tracked in #1665, #2890 and #5809 — this issue focuses on the encoding defect that triggers it, but it's worth noting the impact is amplified by that gap):
- The upload API returns a token as usual (HTTP 200)
- The UI shows a successful upload; the project simply stays at 0 components with no error indication
- `/api/v1/bom/token/{token}` flips `processing` to `false` with no error state
- A `BOM_PROCESSING_FAILED` notification is emitted internally, but reaches nobody unless an alert publisher is explicitly configured
The only evidence is in the server log:
```
ERROR [BomUploadProcessingTask] Failed to process BOM [...]
javax.jdo.JDOFatalUserException: Attempt to store value "pkg:nuget/Vsdbg%20%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF81%FFFFFFC2%FFFFFFAE%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF82%FFFFFFC2%FFFFFFAF...
%20vsdbg@18.10.10709.3%20commit%3A9e45c4be6a7fd2ba9b62caae9dec05e66893094d"
in column "PURLCOORDINATES" that has maximum length of 786. Please correct your data!
at org.datanucleus.api.jdo.JDOAdapter.getJDOExceptionForNucleusException(JDOAdapter.java:681)
at org.datanucleus.api.jdo.JDOPersistenceManager.flush(JDOPersistenceManager.java:2057)
at org.dependencytrack.tasks.BomUploadProcessingTask.processComponents(BomUploadProcessingTask.java:476)
...
```
(Same failure on v5.1.0, where the stack trace goes through `org.dependencytrack.tasks.ImportBomActivity.processBom` and the PostgreSQL column limit is 1024.)
### Root cause
> Note: the following root cause analysis was performed with AI assistance (Fable 5) and verified manually against the packageurl-java 1.5.0 sources and a local Dependency-Track deployment.
The purl in the BOM is only **286 characters** and perfectly valid. It gets inflated ~4.3x by two bugs in `packageurl-java` **1.5.0** (the version pinned by both DT 4.14.4 and DT 5.1.0):
1. **`uriDecode` decodes UTF-8 multi-byte sequences byte-by-byte**
([PackageURL.java#L492-L510](https://github.com/package-url/packageurl-java/blob/packageurl-java-1.5.0/src/main/java/com/github/packageurl/PackageURL.java#L492-L510)):
`%E3%81%AE` (Japanese "の", one 3-byte UTF-8 character) is decoded into three separate chars `\u00E3` + `\u0081` + `\u00AE`, which re-encode to **6** UTF-8 bytes (`C3 A3 C2 81 C2 AE`) instead of 3.
2. **`uriEncode` sign-extends bytes ≥ 0x80**
([PackageURL.java#L443-L457](https://github.com/package-url/packageurl-java/blob/packageurl-java-1.5.0/src/main/java/com/github/packageurl/PackageURL.java#L443-L457)):
```java
builder.append('%');
builder.append(Integer.toHexString(b).toUpperCase()); // byte 0xC3 -> int -61 -> "FFFFFFC3"
```
Every non-ASCII byte becomes `%FFFFFFC3` (9 chars) instead of `%C3` (3 chars).
Net effect: each non-ASCII character in the original purl expands from 9 chars (`%E3%81%AE`) to 54 chars (`%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF81%FFFFFFC2%FFFFFFAE`) when DT canonicalizes the purl to populate `PURLCOORDINATES`, blowing past the column limit. DataNucleus then aborts the whole import transaction.
The encoding rewrite on packageurl-java `master` (upcoming 2.0.0, [StringUtil.percentEncode](https://github.com/package-url/packageurl-java/blob/master/src/main/java/com/github/packageurl/internal/StringUtil.java)) fixes both bugs, but 2.0.0 is not yet released on Maven Central (latest is 1.5.0 from Dec 2023).
Real-world trigger: syft scanning a machine with `vsdbg` — the Microsoft .NET debugger that VS Code's C# extension (and Visual Studio remote debugging) installs **automatically** on any machine where .NET debugging is used, so it is present on a very large number of developer machines, build agents, and remote/embedded targets. Its localized resource DLLs (`~/.vs-debugger/install/{cs,de,es,fr,it,ja,ko,pl,pt-BR,ru,tr,...}/VsDbg.SymbolLocator.resources.dll`) produce nuget purls whose names are localized descriptions in Turkish/Japanese/Russian/etc. One such component in a full-system SBOM kills the entire import.
### Steps to Reproduce
1. Start Dependency-Track:
```yaml
# docker-compose.yml
services:
dtrack-apiserver:
image: dependencytrack/apiserver:4.14.4
environment:
- ALPINE_DATABASE_MODE=embedded
ports:
- "8081:8080"
volumes:
- dtrack-data:/data
volumes:
dtrack-data:
```
2. Save this minimal BOM as `minimal-sbom.json` (single component, real-world purl produced by syft from `vsdbg`'s Japanese resource DLL — 286 chars, well within limits):
```json
{
"bomFormat": "CycloneDX",
"specVersion": "1.7",
"version": 1,
"components": [
{
"type": "library",
"bom-ref": "repro-component",
"name": "Vsdbg のクロスプラットフォーム .NET シンボル ロケーター",
"version": "18.10.10709.3 commit:9e45c4be6a7fd2ba9b62caae9dec05e66893094d",
"purl": "pkg:nuget/Vsdbg%20%E3%81%AE%E3%82%AF%E3%83%AD%E3%82%B9%E3%83%97%E3%83%A9%E3%83%83%E3%83%88%E3%83%95%E3%82%A9%E3%83%BC%E3%83%A0%20.NET%20%E3%82%B7%E3%83%B3%E3%83%9C%E3%83%AB%20%E3%83%AD%E3%82%B1%E3%83%BC%E3%82%BF%E3%83%BC@18.10.10709.3%20commit%3A9e45c4be6a7fd2ba9b62caae9dec05e66893094d"
}
]
}
```
3. Upload it (via UI, or API):
```bash
API=http://localhost:8081
curl -s -X POST "$API/api/v1/user/forceChangePassword" \
--data-urlencode "username=admin" --data-urlencode "password=admin" \
--data-urlencode "newPassword=Repro123!" --data-urlencode "confirmPassword=Repro123!"
JWT=$(curl -s -X POST "$API/api/v1/user/login" \
--data-urlencode "username=admin" --data-urlencode "password=Repro123!")
curl -s -X PUT "$API/api/v1/bom" \
-H "Authorization: Bearer $JWT" -H 'Content-Type: application/json' \
-d "{\"projectName\":\"purl-repro\",\"projectVersion\":\"1.0\",\"autoCreate\":true,\"bom\":\"$(base64 -w0 minimal-sbom.json)\"}"
```
4. Observe:
- Upload returns HTTP 200 with a token
- Project `purl-repro` exists but has 0 components; no error visible in UI or API
- `docker compose logs dtrack-apiserver` contains the `JDOFatalUserException` / `PURLCOORDINATES` error above
### Alternative: full end-to-end reproduction from scratch
Instead of step 2, generate the BOM the same way it occurs in the wild — download `vsdbg` using Microsoft's official install script (the same one VS Code runs) and scan it with [syft](https://github.com/anchore/syft):
```bash
# download vsdbg (installs the localized resource DLLs)
curl -sSL https://aka.ms/getvsdbgsh | bash /dev/stdin -v latest -l ./vsdbg-install
# generate a CycloneDX SBOM from it
syft dir:./vsdbg-install -o cyclonedx-json=minimal-sbom.json
# show the offending purls (one per localization)
jq -r '.components[] | select(.purl != null and (.purl | test("%[89A-F][0-9A-F]"))) | .purl' minimal-sbom.json
```
Uploading that BOM in step 3 fails identically.
### Expected Behavior
- A valid purl with percent-encoded UTF-8 characters should be canonicalized correctly (per the purl spec) and stored without inflation — i.e. don't corrupt `%E3%81%AE` into `%FFFFFFC3%FFFFFFA3%FFFFFFC2%FFFFFF81%FFFFFFC2%FFFFFFAE`.
- A single malformed/oversized component should not abort the entire BOM import — skip it with a warning, or fail visibly (feedback gap already tracked in #1665 / #2890).
## Workarounds we found
- Exclude the vsdbg directory from syft scans: `--exclude '**/.vs-debugger/**'`
- Or strip affected components from an existing BOM: `jq` filter removing components whose purl matches `%[89A-F][0-9A-F]` percent-encodings
- Configure a `BOM_PROCESSING_FAILED` alert publisher to at least make the failure visible
### Dependency-Track Version
4.x
### Browser
Microsoft Edge
### Checklist
- [x] I have read and understand the [contributing guidelines](https://github.com/DependencyTrack/dependency-track/blob/main/CONTRIBUTING.md#filing-issues)
- [x] I have checked the [existing issues](https://github.com/DependencyTrack/dependency-track/issues) for whether this defect was already reported
Contributor guide
Research direction
Start at org.dependencytrack.tasks.BomUploadProcessingTask.processComponents, and compare the v5.1.0 path through ImportBomActivity.processBom. Reproduce the failure with minimal-sbom.json and inspect the pinned packageurl-java 1.5.0 behavior. Done means percent-encoded UTF-8 purls are stored without inflation and the BOM import no longer aborts on this component.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, postgresql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100