jenkinsci / jenkinsci/github-branch-source-plugin
[JENKINS-73242] GitHub Organization scan fails when repository is DMCA'd
- Dominant language
- Java
- Stars
- 217
- Forks
- 398
- Avg merge
- 30m
- Merged PRs (30d)
- 1
Description
Scanning a github organization/user fails if a repository is blocked due to a DMCA takedown.
Here is an excerpt from the end of my scan log:
Ignoring krypton
20:13:09 Skipping repository krypton because it is a fork
Ignoring laptop-setup-ubuntu-18.04
20:13:09 Skipping repository laptop-setup-ubuntu-18.04 because it is a fork
Ignoring LearningDesignPattern
20:13:09 Skipping repository LearningDesignPattern because it is a fork
Ignoring Ledger
20:13:10 Skipping repository Ledger because it is a fork
Ignoring LiquidBurner
20:13:10 Skipping repository LiquidBurner because it is a fork
ERROR: [Sun Jun 02 20:13:10 EDT 2024] Could not fetch sources from navigator org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator@7010654c
[Sun Jun 02 20:13:10 EDT 2024] Finished organization scan. Scan took 17 sec FATAL: Failed to recompute children of solonovamax
org.kohsuke.github.HttpException: {"message":"Repository access blocked","block":{"reason":"dmca","created_at":"2023-05-08T22:39:57Z","html_url":"
https://github.com/github/dmca/blob/master/2023/05/2023-05-04-nintendo-4.md
"}}
at PluginClassLoader for github-api//org.kohsuke.github.GitHubConnectorResponseErrorHandler$1.onError(GitHubConnectorResponseErrorHandler.java:62)
at PluginClassLoader for github-api//org.kohsuke.github.GitHubClient.detectKnownErrors(GitHubClient.java:504)
at PluginClassLoader for github-api//org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:464)
at PluginClassLoader for github-api//org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:427)
at PluginClassLoader for github-api//org.kohsuke.github.Requester.fetchInto(Requester.java:102)
at PluginClassLoader for github-api//org.kohsuke.github.GHRepository.populate(GHRepository.java:3599)
at PluginClassLoader for github-api//org.kohsuke.github.GHRepository.getSource(GHRepository.java:2979)
at PluginClassLoader for github-branch-source//org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:1068)
at PluginClassLoader for branch-api//jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:530)
at PluginClassLoader for cloudbees-folder//com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:269)
at PluginClassLoader for cloudbees-folder//com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:167)
at PluginClassLoader for branch-api//jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:915)
at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:446)
Finished: FAILURE
The repository in question is most likely one of the following:
These repositories cannot be removed through github. The DMCA was automatic and affected any users which had forked this repository (of which there were many, including me), so being unable to scan repositories due to this is quite annoying.
this is my first time reporting an issue for jenkins, so lmk if I left out any information (tho I think I got everything)
---
Originally reported by solonovamax, imported from: GitHub Organization scan fails when repository is DMCA'd
Raw content of original issue
Scanning a github organization/user fails if a repository is blocked due to a DMCA takedown.
Here is an excerpt from the end of my scan log:
Ignoring krypton
20:13:09 Skipping repository krypton because it is a fork
Ignoring laptop-setup-ubuntu-18.04
20:13:09 Skipping repository laptop-setup-ubuntu-18.04 because it is a fork
Ignoring LearningDesignPattern
20:13:09 Skipping repository LearningDesignPattern because it is a fork
Ignoring Ledger
20:13:10 Skipping repository Ledger because it is a fork
Ignoring LiquidBurner
20:13:10 Skipping repository LiquidBurner because it is a fork
ERROR: [Sun Jun 02 20:13:10 EDT 2024] Could not fetch sources from navigator org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator@7010654c
[Sun Jun 02 20:13:10 EDT 2024] Finished organization scan. Scan took 17 sec FATAL: Failed to recompute children of solonovamax
org.kohsuke.github.HttpException: {"message":"Repository access blocked","block":{"reason":"dmca","created_at":"2023-05-08T22:39:57Z","html_url":"
https://github.com/github/dmca/blob/master/2023/05/2023-05-04-nintendo-4.md
"}}
at PluginClassLoader for github-api//org.kohsuke.github.GitHubConnectorResponseErrorHandler$1.onError(GitHubConnectorResponseErrorHandler.java:62)
at PluginClassLoader for github-api//org.kohsuke.github.GitHubClient.detectKnownErrors(GitHubClient.java:504)
at PluginClassLoader for github-api//org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:464)
at PluginClassLoader for github-api//org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:427)
at PluginClassLoader for github-api//org.kohsuke.github.Requester.fetchInto(Requester.java:102)
at PluginClassLoader for github-api//org.kohsuke.github.GHRepository.populate(GHRepository.java:3599)
at PluginClassLoader for github-api//org.kohsuke.github.GHRepository.getSource(GHRepository.java:2979)
at PluginClassLoader for github-branch-source//org.jenkinsci.plugins.github_branch_source.GitHubSCMNavigator.visitSources(GitHubSCMNavigator.java:1068)
at PluginClassLoader for branch-api//jenkins.branch.OrganizationFolder.computeChildren(OrganizationFolder.java:530)
at PluginClassLoader for cloudbees-folder//com.cloudbees.hudson.plugins.folder.computed.ComputedFolder.updateChildren(ComputedFolder.java:269)
at PluginClassLoader for cloudbees-folder//com.cloudbees.hudson.plugins.folder.computed.FolderComputation.run(FolderComputation.java:167)
at PluginClassLoader for branch-api//jenkins.branch.OrganizationFolder$OrganizationScan.run(OrganizationFolder.java:915)
at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:446)
Finished: FAILURE
The repository in question is most likely one of the following:
These repositories cannot be removed through github. The DMCA was automatic and affected any users which had forked this repository (of which there were many, including me), so being unable to scan repositories due to this is quite annoying.
this is my first time reporting an issue for jenkins, so lmk if I left out any information (tho I think I got everything)
environment
```
Jenkins: 2.460
OS: Linux - 6.1.0-10-amd64
Java: 17.0.11 - Debian (OpenJDK 64-Bit Server VM)
---
adoptopenjdk:1.5
antisamy-markup-formatter:162.v0e6ec0fcfcf6
apache-httpcomponents-client-4-api:4.5.14-208.v438351942757
asm-api:9.7-33.v4d23ef79fcc8
bootstrap5-api:5.3.3-1
bouncycastle-api:2.30.1.78.1-233.vfdcdeb_0a_08a_a_
branch-api:2.1169.va_f810c56e895
build-name-setter:2.4.2
build-timeout:1.32
caffeine-api:3.1.8-133.v17b_1ff2e0599
checks-api:2.2.0
cloudbees-disk-usage-simple:203.v3f46a_7462b_1a_
cloudbees-folder:6.942.vb_43318a_156b_2
command-launcher:107.v773860566e2e
commons-lang3-api:3.13.0-62.v7d18e55f51e2
commons-text-api:1.11.0-109.vfe16c66636eb_
config-file-provider:973.vb_a_80ecb_9a_4d0
configuration-as-code:1810.v9b_c30a_249a_4c
credentials:1337.v60b_d7b_c7b_c9f
credentials-binding:677.vdc9d38cb_254d
dark-theme:465.vd8b_419b_0728f
dashboard-view:2.508.va_74654f026d1
data-tables-api:2.0.7-1
discord-notifier:241.v448b_ccd0c0d6
display-url-api:2.204.vf6fddd8a_8b_e9
durable-task:555.v6802fe0f0b_82
echarts-api:5.5.0-1
eddsa-api:0.3.0-4.v84c6f0f4969e
email-ext:1814.v404722f34263
embeddable-build-status:487.va_0ef04c898a_2
font-awesome-api:6.5.2-1
git:5.2.2
git-client:4.7.0
git-parameter:0.9.19
git-server:126.v0d945d8d2b_39
github:1.39.0
github-api:1.318-461.v7a_c09c9fa_d63
github-branch-source:1789.v5b_0c0cea_18c3
github-pullrequest:0.7.0
gitlab-api:5.3.0-91.v1f9a_fda_d654f
gitlab-plugin:1.8.1
gradle:2.12
gradle-daemon:0.1.0
gson-api:2.11.0-41.v019fcf6125dc
instance-identity:185.v303dc7c645f9
ionicons-api:74.v93d5eb_813d5f
jackson2-api:2.17.0-379.v02de8ec9f64c
jakarta-activation-api:2.1.3-1
jakarta-mail-api:2.1.3-1
javadoc:243.vb_b_503b_b_45537
javax-activation-api:1.2.0-7
javax-mail-api:1.6.2-10
jaxb:2.3.9-1
jdk-tool:73.vddf737284550
jersey2-api:2.42-147.va_28a_44603b_d5
jjwt-api:0.11.5-112.ve82dfb_224b_a_d
joda-time-api:2.12.7-29.v5a_b_e3a_82269a_
jquery3-api:3.7.1-2
jsch:0.2.16-86.v42e010d9484b_
json-api:20240303-41.v94e11e6de726
json-path-api:2.9.0-58.v62e3e85b_a_655
junit:1265.v65b_14fa_f12f0
locale:495.v4606fdb_134ef
lockable-resources:1255.vf48745da_35d0
mailer:472.vf7c289a_4b_420
material-theme:0.5.2-rc100.6121925fe229
matrix-auth:3.2.2
matrix-project:831.v084e85a_b_4ea_d
maven-plugin:3.23
metrics:4.2.21-451.vd51df8df52ec
mina-sshd-api-common:2.12.1-101.v85b_e08b_780dd
mina-sshd-api-core:2.12.1-101.v85b_e08b_780dd
monitoring:1.99.0
okhttp-api:4.11.0-172.vda_da_1feeb_c6e
pam-auth:1.11
pipeline-build-step:540.vb_e8849e1a_b_d8
pipeline-github-lib:61.v629f2cc41d83
pipeline-graph-analysis:216.vfd8b_ece330ca_
pipeline-groovy-lib:710.v4b_94b_077a_808
pipeline-input-step:495.ve9c153f6067b_
pipeline-milestone-step:119.vdfdc43fc3b_9a_
pipeline-model-api:2.2198.v41dd8ef6dd56
pipeline-model-definition:2.2198.v41dd8ef6dd56
pipeline-model-extensions:2.2198.v41dd8ef6dd56
pipeline-rest-api:2.34
pipeline-stage-step:312.v8cd10304c27a_
pipeline-stage-tags-metadata:2.2198.v41dd8ef6dd56
pipeline-stage-view:2.34
pipeline-utility-steps:2.16.2
plain-credentials:182.v468b_97b_9dcb_8
plugin-util-api:4.1.0
prism-api:1.29.0-15
prometheus:770.ve5e758db_606e
rebuild:332.va_1ee476d8f6d
resource-disposer:0.23
role-strategy:727.vd344b_eec783d
scm-api:690.vfc8b_54395023
script-security:1341.va_2819b_414686
simple-theme-plugin:176.v39740c03a_a_f5
snakeyaml-api:2.2-111.vc6598e30cc65
ssh-credentials:337.v395d2403ccd4
ssh-slaves:2.968.v6f8823c91de4
sshd:3.322.v159e91f6a_550
structs:337.v1b_04ea_4df7c8
theme-manager:215.vc1ff18d67920
timestamper:1.27
token-macro:400.v35420b_922dcb_
trilead-api:2.147.vb_73cc728a_32e
variant:60.v7290fc0eb_b_cd
workflow-aggregator:596.v8c21c963d92d
workflow-api:1316.v33eb_726c50b_a_
workflow-basic-steps:1058.vcb_fc1e3a_21a_9
workflow-cps:3894.3896.vca_2c931e7935
workflow-durable-task-step:1353.v1891a_b_01da_18
workflow-job:1426.v2ecb_a_a_42fd46
workflow-multibranch:791.v28fb_f74dfca_e
workflow-scm-step:427.v4ca_6512e7df1
workflow-step-api:657.v03b_e8115821b_
workflow-support:907.v6713a_ed8a_573
ws-cleanup:0.46
```
Contributor guide
Assessment
This issue has not been assessed yet.