docToolchain / docToolchain/docToolchain
Configure project : dependency resolution with custom JFrog artifactory URL not working
- Dominant language
- Groovy
- Stars
- 864
- Forks
- 240
- Avg merge
- 2m
- Merged PRs (30d)
- 2
Description
**Describe the bug**
We want to perform builds using Doctoolchain in our company. We use our own JFrog instance, which is only accessible from the intranet. Therefore, all Gradle dependencies for the Doctoolchain build should be loaded from the JFrog Artifactory. To achieve this, we tried the solution described in the Doctoolchain documentation, [https://doctoolchain.org/docToolchain/v2.0.x/020_tutorial/160_EnterpriseTipsAndTricks.html#_dependencies](https://doctoolchain.org/docToolchain/v2.0.x/020_tutorial/160_EnterpriseTipsAndTricks.html#_dependencies). However, this approach does not work.
In Gradle, the buildscript block of the script plugin cannot be overridden from the `init.d` file ([https://stackoverflow.com/questions/56283254/overwrite-repository-definition-in-gradle-subproject](https://stackoverflow.com/questions/56283254/overwrite-repository-definition-in-gradle-subproject)). While I can specify my JFrog repository for the buildscript blocks using the `mavenRepository`, `mavenUsername`, and `mavenPassword` properties, there are two files where `mavenCentral()` appears before the `maven` block with these properties. Due to Gradle's logic, this causes the dependency resolution to first look for dependencies in `mavenCentral()`. This results in our build pipeline, which does not have internet access, failing because it cannot reach the public `mavenCentral`.
See:
`AsciiDocBasics.gradle` & `exportMarkdown.gradle`:
```groovy
repositories {
mavenCentral() <---
maven {
credentials {
username mavenUsername
password mavenPassword
}
url mavenRepository
}
}
```
Since there is no way to modify the buildscript blocks from the outside, I would suggest changing the order as a first step. Subsequently, I would refactor the Gradle scripts so that all buildscript blocks are removed. I will submit a pull request to address the issue.
**To Reproduce**
Steps to reproduce the behavior:
1. set custom Artifactory URL and Credentials like mentioned in the doc or with the properties like mavenRepository, etc: [https://doctoolchain.org/docToolchain/v2.0.x/020_tutorial/160_EnterpriseTipsAndTricks.html#_dependencies](https://doctoolchain.org/docToolchain/v2.0.x/020_tutorial/160_EnterpriseTipsAndTricks.html#_dependencies)
2. Block [mavenCentral](https://mvnrepository.com/repos/central) and repo.maven.apache.org
3. ./gradlew build
4. See error
**Expected behavior**
I expect the gradle build to work by providing my JFrog Informationen through `mavenRepository`, `mavenUsername`, and `mavenPassword` and `depsMavenRepository`, `depsMavenUsername`, and `depsMavenPassword` to work in a closed environment without access to the internet.
**Screenshots**
Logs:
```
compiling script '/home/s_appl/.doctoolchain/docToolchain-3.4.0/scripts/AsciiDocBasics.gradle' using SubsetScriptTransformer.
FAILURE: Build failed with an exception.
* Where:
Build file '/home/s_appl/.doctoolchain/docToolchain-3.4.0/build.gradle' line: 98
* What went wrong:
A problem occurred evaluating root project 'docToolchain'.
> Could not resolve all artifacts for configuration 'classpath'.
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm:4.0.0-alpha.1.
Required by:
unspecified:unspecified:unspecified
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm:4.0.0-alpha.1.
> Could not get resource 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm/4.0.0-alpha.1/asciidoctor-gradle-jvm-4.0.0-alpha.1.pom'.
> Could not GET 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm/4.0.0-alpha.1/asciidoctor-gradle-jvm-4.0.0-alpha.1.pom'.
> Connect to repo.maven.apache.org:443 [repo.maven.apache.org/146.75.120.215] failed: Connect timed out
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm-pdf:4.0.0-alpha.1.
Required by:
unspecified:unspecified:unspecified
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm-pdf:4.0.0-alpha.1.
> Could not get resource 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm-pdf/4.0.0-alpha.1/asciidoctor-gradle-jvm-pdf-4.0.0-alpha.1.pom'.
> Could not GET 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm-pdf/4.0.0-alpha.1/asciidoctor-gradle-jvm-pdf-4.0.0-alpha.1.pom'.
> Connect to repo.maven.apache.org:443 [repo.maven.apache.org/146.75.120.215] failed: Connect timed out
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm-gems:4.0.0-alpha.1.
Required by:
unspecified:unspecified:unspecified
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm-gems:4.0.0-alpha.1.
> Could not get resource 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm-gems/4.0.0-alpha.1/asciidoctor-gradle-jvm-gems-4.0.0-alpha.1.pom'.
> Could not GET 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm-gems/4.0.0-alpha.1/asciidoctor-gradle-jvm-gems-4.0.0-alpha.1.pom'.
> Connect to repo.maven.apache.org:443 [repo.maven.apache.org/146.75.120.215] failed: Connect timed out
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm-slides:4.0.0-alpha.1.
Required by:
unspecified:unspecified:unspecified
> Could not resolve org.asciidoctor:asciidoctor-gradle-jvm-slides:4.0.0-alpha.1.
> Could not get resource 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm-slides/4.0.0-alpha.1/asciidoctor-gradle-jvm-slides-4.0.0-alpha.1.pom'.
> Could not GET 'https://repo.maven.apache.org/maven2/org/asciidoctor/asciidoctor-gradle-jvm-slides/4.0.0-alpha.1/asciidoctor-gradle-jvm-slides-4.0.0-alpha.1.pom'.
> Connect to repo.maven.apache.org:443 [repo.maven.apache.org/146.75.120.215] failed: Connect timed out
```
**Configuration**
- docToolchain version: 3.4.0
- OS: [e.g. Windows, Linux, ...] Windows
- Browser [e.g. chrome, safari] -
- Relevant SDK Versions [e.g. JDK 1.8, ...] JDK 17.0.12
**Additional context**
init.d gradle content:
```
ext {
jfrogRepoUrl = System.getenv('REPO_VIRTUAL_MAVEN_REGISTRY') ?: null
jfrogRepoUser = System.getenv('REPO_USER') ?: null
jfrogRepoPw = System.getenv('REPO_TOKEN') ?: null
}
if (jfrogRepoUrl == null) {
println "env REPO_VIRTUAL_MAVEN_REGISTRY is not defined."
}
if (jfrogRepoUser == null) {
println "env REPO_USER is not defined."
}
if (jfrogRepoPw == null) {
println "env REPO_TOKEN is not defined."
}
if (jfrogRepoUrl != null && jfrogRepoUser != null && jfrogRepoPw != null) {
allprojects {
repositories {
maven {
url "${jfrogRepoUrl}"
credentials {
username = "${jfrogRepoUser}"
password = "${jfrogRepoPw}"
}
}
}
}
settingsEvaluated { settings ->
settings.pluginManagement {
println "removing plugin management repositories: t" + repositories.getNames()
repositories.clear()
// add jfrog artifactory as repository
repositories {
maven {
url "${jfrogRepoUrl}"
credentials {
username = "${jfrogRepoUser}"
password = "${jfrogRepoPw}"
}
}
}
println "new plugin management repositories: " + repositories.getNames()
}
settings.dependencyResolutionManagement {
println "removing dependency resolution management repositories: " + repositories.getNames()
repositories.clear()
// add jfrog artifactory as repository
repositories {
maven {
url "${jfrogRepoUrl}"
credentials {
username = "${jfrogRepoUser}"
password = "${jfrogRepoPw}"
}
}
}
println "new dependency resolution management repositories: " + repositories.getNames()
}
}
} else {
println "JFrog not configured because REPO_VIRTUAL_MAVEN_REGISTRY or REPO_USER or REPO_TOKEN was not present in the env."
}
// needed for doctoolchain because we cant override the buildscript repositories of doctoolchain
gradle.projectsLoaded {
rootProject.allprojects {
if (project.hasProperty("mavenRepository")) {
project.setProperty("mavenRepository", "${jfrogRepoUrl}")
println "set mavenRepository property"
}
if (project.hasProperty("mavenUsername")) {
project.setProperty("mavenUsername", "${jfrogRepoUser}")
println "set mavenUsername property"
}
if (project.hasProperty("mavenPassword")) {
project.setProperty("mavenPassword", "${jfrogRepoPw}")
println "set mavenPassword property"
}
if (project.hasProperty("depsMavenRepository")) {
project.setProperty("depsMavenRepository", "${jfrogRepoUrl}")
println "set depsMavenRepository property"
}
if (project.hasProperty("depsMavenUsername")) {
project.setProperty("depsMavenUsername", "${jfrogRepoUser}")
println "set depsMavenUsername property"
}
if (project.hasProperty("depsMavenPassword")) {
project.setProperty("depsMavenPassword", "${jfrogRepoPw}")
println "set depsMavenPassword property"
}
}
}
```
**Find a good headline**
Now it is time to find a good headline/subject for the problem.
Please prefix it with the name of the task which fails.
Example:
exportContributors throws exception when ...
Contributor guide
Research direction
Start with AsciiDocBasics.gradle and exportMarkdown.gradle, then reproduce the failure with ./gradlew build using the listed Maven properties while Maven Central is blocked. Verify the repository configuration and dependency-resolution behavior; done means the build resolves all dependencies through the configured Artifactory without accessing Maven Central.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- groovy
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100