DependencyTrack / DependencyTrack/dependency-track

The MavenMetaAnalyzer task fails due to invalid URLs

Open
#3,566 2 comments 0 reactions 0 assignees View on GitHub
defect good first issue hacktoberfest p2 size/S
Dominant language
Java
Stars
4.2k
Forks
811
Avg merge
8h 39m
Merged PRs (30d)
237

Description

### Current Behavior

In the logs, I can see that the MavenMetaAnalyzer task tails due to invalid URLs formatted with [parts of the PURL of a component](https://github.com/DependencyTrack/dependency-track/blob/333c56d44a7db3447bb1e7126a05b8df6ea717b1/src/main/java/org/dependencytrack/tasks/repositories/MavenMetaAnalyzer.java#L79):
```
compose-dtrack-apiserver-1 | 2024-03-18 08:51:29,133 INFO [InternalAnalysisTask] Starting internal analysis task
compose-dtrack-apiserver-1 | 2024-03-18 08:51:29,133 INFO [InternalAnalysisTask] Analyzing 171 component(s)
compose-dtrack-apiserver-1 | [Fatal Error] :1:10: DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
compose-dtrack-apiserver-1 | 2024-03-18 08:51:31,639 ERROR [MavenMetaAnalyzer] Request failure
compose-dtrack-apiserver-1 | org.xml.sax.SAXParseException: DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.
compose-dtrack-apiserver-1 | at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
compose-dtrack-apiserver-1 | at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
compose-dtrack-apiserver-1 | at java.xml/javax.xml.parsers.DocumentBuilder.parse(Unknown Source)
compose-dtrack-apiserver-1 | at org.dependencytrack.tasks.repositories.MavenMetaAnalyzer.analyze(MavenMetaAnalyzer.java:86)
compose-dtrack-apiserver-1 | at org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask.analyze(RepositoryMetaAnalyzerTask.java:177)
compose-dtrack-apiserver-1 | at org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask.lambda$analyze$0(RepositoryMetaAnalyzerTask.java:121)
compose-dtrack-apiserver-1 | at io.github.resilience4j.retry.Retry.lambda$decorateCallable$5(Retry.java:237)
compose-dtrack-apiserver-1 | at io.github.resilience4j.retry.Retry.executeCallable(Retry.java:373)
compose-dtrack-apiserver-1 | at org.dependencytrack.util.CacheStampedeBlocker.readThroughOrPopulateCache(CacheStampedeBlocker.java:201)
compose-dtrack-apiserver-1 | at org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask.analyze(RepositoryMetaAnalyzerTask.java:126)
compose-dtrack-apiserver-1 | at org.dependencytrack.tasks.repositories.RepositoryMetaAnalyzerTask.inform(RepositoryMetaAnalyzerTask.java:91)
compose-dtrack-apiserver-1 | at alpine.event.framework.BaseEventService.lambda$publish$0(BaseEventService.java:110)
compose-dtrack-apiserver-1 | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
compose-dtrack-apiserver-1 | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
compose-dtrack-apiserver-1 | at java.base/java.lang.Thread.run(Unknown Source)
```

(**NB:** The DOCTYPE probably stems from a plain HTTP response for a 404 page, but this is just a guess since the URL isn't logged)

It is however impossible to know which component(s) that cause this since the component name isn't logged in the analyze() method. If that would have been logged, one could have inspected+corrected the PURL of the component in the DB and error-traced the chain that led to the invalid PURL.

My suggestion is that:

1. Something along the lines of "Analyzing component " + component gets logged in the analyze() method, for traceability
2. The URL is validated before it gets passed to processHttpRequest

### Steps to Reproduce

Hard to specify, since DTrack doesn't log which component is the root of the cause.

### Expected Behavior

1. The generated URL gets validated before it gets used. If invalid, a warning along the lines of `"Invalid url: " + url` gets logged
2. Each time the MavenMetaAnalyzer.analyze() method is called, `"Analyzing " + component` is logged for traceability

### Dependency-Track Version

4.10.1

### Dependency-Track Distribution

Container Image

### Database Server

PostgreSQL

### Database Server Version

13.13

### Browser

N/A

### Checklist

- [X] I have read and understand the [contributing guidelines](https://github.com/DependencyTrack/dependency-track/blob/master/CONTRIBUTING.md#filing-issues)
- [X] I have checked the [existing issues](https://github.com/DependencyTrack/dependency-track/issues) for whether this defect was already reported

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.