arduino / arduino/Arduino

Can't download new board support behind proxy

Open
#9,341 0 comments 0 reactions 0 assignees View on GitHub
Component: Board/Lib Manager Component: Proxy Type: Bug
Dominant language
Java
Stars
14.6k
Forks
7k
PR merge metrics
No merged PRs in 30d

Description

I can't download new board support behind a proxy, using Arduino 1.8.10 under Ubuntu 18.04 LTS.

I'm behind my university's proxy that uses authentication.

First I tried the automatic proxy setup with the according script which fails the same way as using the manual http proxy setting to download the `package_index.json`:

```
2019-10-21T13:45:31.765Z DEBUG c.a.u.n.HttpConnectionManager:125 [cc.arduino.packages.discoverers.serial.SerialDiscovery] Using proxy HTTP @ proxy.hs-karlsruhe.de/193.196.64.2:8888
2019-10-21T13:45:32.185Z INFO c.a.u.n.HttpConnectionManager:148 [cc.arduino.packages.discoverers.serial.SerialDiscovery] Connect to https://builder.arduino.cc/builder/v1/boards/0x8087/0x0AB6, method=GET, request id=34F2F0F71F254597
2019-10-21T13:45:32.242Z WARN p.a.h.BoardCloudResolver:79 [cc.arduino.packages.discoverers.serial.SerialDiscovery] Error during get board information by vid, pid
java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2142) ~[?:1.8.0_191]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) ~[?:1.8.0_191]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:150) ~[arduino-core.jar:?]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:112) ~[arduino-core.jar:?]
at processing.app.helpers.BoardCloudResolver.getBoardBy(BoardCloudResolver.java:61) [arduino-core.jar:?]
at cc.arduino.packages.discoverers.serial.SerialDiscovery.forceRefresh(SerialDiscovery.java:204) [arduino-core.jar:?]
at cc.arduino.packages.discoverers.serial.SerialDiscovery$1.run(SerialDiscovery.java:97) [arduino-core.jar:?]
at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_191]
at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_191]
2019-10-21T13:45:39.745Z INFO c.a.c.p.ContributionInstaller:305 [ContributionManager Update Thread] Start download and signature check of=[https://downloads.arduino.cc/packages/package_index.json]
2019-10-21T13:45:39.763Z INFO c.a.u.n.FileDownloaderCache:92 [ContributionManager Update Thread] Cache folder /home/client1ubuntu/.arduino15/cache
2019-10-21T13:45:39.854Z INFO c.a.u.n.FileDownloaderCache:149 [ContributionManager Update Thread] Get file cached is expire true, exist false, info FileCached{eTag='null', lastETag='null', remoteURL='https://downloads.arduino.cc/packages/package_index.json', localPath='/home/client1ubuntu/.arduino15/cache/downloads.arduino.cc/packages/package_index.json', md5='null', createdAt='2019-10-21T15:45:39.804', cacheControl=null}
2019-10-21T13:45:39.859Z DEBUG c.a.u.n.HttpConnectionManager:125 [ContributionManager Update Thread] Using proxy HTTP @ proxy.hs-karlsruhe.de/193.196.64.2:8888
2019-10-21T13:45:39.878Z INFO c.a.u.n.HttpConnectionManager:148 [ContributionManager Update Thread] Connect to https://downloads.arduino.cc/packages/package_index.json, method=HEAD, request id=B04AAD63855047CB
2019-10-21T13:45:39.908Z ERROR c.a.u.n.FileDownloader:199 [ContributionManager Update Thread] The request stop
java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2142) ~[?:1.8.0_191]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) ~[?:1.8.0_191]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:150) ~[arduino-core.jar:?]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:106) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloaderCache.updateCacheInfo(FileDownloaderCache.java:184) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloaderCache.getFileCached(FileDownloaderCache.java:153) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.downloadFile(FileDownloader.java:167) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.download(FileDownloader.java:129) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:142) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.downloadIndexAndSignature(DownloadableContributionsDownloader.java:160) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ContributionInstaller.updateIndex(ContributionInstaller.java:306) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:146) ~[pde.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
2019-10-21T13:45:39.913Z ERROR c.a.c.DownloadableContributionsDownloader:176 [ContributionManager Update Thread] Cannot download the package index from https://downloads.arduino.cc/packages/package_index.json the package will be discard
java.lang.Exception: Error downloading https://downloads.arduino.cc/packages/package_index.json
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:144) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.downloadIndexAndSignature(DownloadableContributionsDownloader.java:160) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ContributionInstaller.updateIndex(ContributionInstaller.java:306) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:146) ~[pde.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2142) ~[?:1.8.0_191]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) ~[?:1.8.0_191]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:150) ~[arduino-core.jar:?]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:106) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloaderCache.updateCacheInfo(FileDownloaderCache.java:184) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloaderCache.getFileCached(FileDownloaderCache.java:153) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.downloadFile(FileDownloader.java:167) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.download(FileDownloader.java:129) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:142) ~[arduino-core.jar:?]
... 4 more
2019-10-21T13:45:39.918Z ERROR c.a.c.p.ContributionInstaller:308 [ContributionManager Update Thread] Error downloading https://downloads.arduino.cc/packages/package_index.json
java.lang.Exception: Error downloading https://downloads.arduino.cc/packages/package_index.json
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:144) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.downloadIndexAndSignature(DownloadableContributionsDownloader.java:160) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ContributionInstaller.updateIndex(ContributionInstaller.java:306) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onUpdatePressed$1(ContributionManagerUI.java:146) ~[pde.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.0 407 Proxy Authentication Required"
at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2142) ~[?:1.8.0_191]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) ~[?:1.8.0_191]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:150) ~[arduino-core.jar:?]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:106) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloaderCache.updateCacheInfo(FileDownloaderCache.java:184) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloaderCache.getFileCached(FileDownloaderCache.java:153) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.downloadFile(FileDownloader.java:167) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.download(FileDownloader.java:129) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:142) ~[arduino-core.jar:?]
... 4 more
2019-10-21T13:45:39.922Z INFO c.a.c.p.ContributionInstaller:314 [ContributionManager Update Thread] Downloaded package index URL=[https://downloads.arduino.cc/packages/package_index.json]
2019-10-21T13:45:39.924Z INFO c.a.c.p.ContributionInstaller:324 [ContributionManager Update Thread] Check unknown files. Additional package index folder files=[package_index.json], Additional package index url downloaded=[]
```

Then I tried the socks-proxy which worked to download the `package_index.json` but then fails when I try to download the board-support libraries with the error:
```
2019-10-21T14:13:18.100Z INFO c.a.u.n.FileDownloaderCache:131 [ContributionManager Install Thread] The cache is not enable.
2019-10-21T14:13:18.108Z DEBUG c.a.u.n.HttpConnectionManager:125 [ContributionManager Install Thread] Using proxy DIRECT
2019-10-21T14:13:18.109Z INFO c.a.u.n.HttpConnectionManager:148 [ContributionManager Install Thread] Connect to https://github.com/arduino/ArduinoCore-arc32/archive/2.0.3.zip, method=GET, request id=5514BA7A04A14F91
2019-10-21T14:13:23.118Z ERROR c.a.u.n.FileDownloader:194 [ContributionManager Install Thread] The request went in socket timeout
java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_191]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_191]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_191]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_191]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_191]
at java.net.Socket.connect(Socket.java:589) ~[?:1.8.0_191]
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:666) ~[?:1.8.0_191]
at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[?:1.8.0_191]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~[?:1.8.0_191]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsClient.(HttpsClient.java:264) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:367) ~[?:1.8.0_191]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191) ~[?:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1199) ~[?:1.8.0_191]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1050) ~[?:1.8.0_191]
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177) ~[?:1.8.0_191]
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) ~[?:1.8.0_191]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:150) ~[arduino-core.jar:?]
at cc.arduino.utils.network.HttpConnectionManager.makeConnection(HttpConnectionManager.java:106) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.openConnectionAndFillTheFile(FileDownloader.java:234) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.downloadFile(FileDownloader.java:182) ~[arduino-core.jar:?]
at cc.arduino.utils.network.FileDownloader.download(FileDownloader.java:129) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:142) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:78) ~[arduino-core.jar:?]
at cc.arduino.contributions.DownloadableContributionsDownloader.download(DownloadableContributionsDownloader.java:60) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ContributionInstaller.install(ContributionInstaller.java:105) ~[arduino-core.jar:?]
at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onInstallPressed$2(ContributionManagerUI.java:172) ~[pde.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
```

Contributor guide

Open the contributing guide

Research direction

Start with cc.arduino.utils.network.HttpConnectionManager and the download flow through FileDownloader and ContributionInstaller, using the reported authenticated HTTP and SOCKS proxy scenarios. Reproduce the 407 and socket-timeout failures; done means board support and package_index.json downloads work through the configured proxy.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.