RipMeApp / RipMeApp/ripme

Socks proxy isn't used (by the looks of things)

Open
#1,663 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
4.1k
Forks
673
PR merge metrics
No merged PRs in 30d

Description

  • Ripme version:1.7.92

  • Java version: openjdk version "11.0.7" 2020-04-14
    OpenJDK Runtime Environment (build 11.0.7+10-post-Ubuntu-3ubuntu1)
    OpenJDK 64-Bit Server VM (build 11.0.7+10-post-Ubuntu-3ubuntu1, mixed mode, sharing)

  • Operating system: Ubuntu 20.04 LTS x86_64

  • Exact URL you were trying to rip when the problem occurred: Any URL which is blocked without my proxy.

  • Please include any additional information about how to reproduce the problem: Running tor proxy locally.

This is getting in the way of two new ripper PRs I have that I think work (but can't test)

Expected Behavior

Use socks proxy provided in the config (proxy.socks = 127.0.0.1:9050). Working for loading the same urls in a browser. I also tried a few other random proxies from the internet and they didn't work either.

Actual Behavior

Always fails to load after 1 attempt (Same as without a proxy in the config)

java.io.IOException: Failed to load https://xhamster.com/videos/brazzers-busty-big-booty-milf-lisa-ann-fucks-her-masseur-1492828 after 1 attempts
	at com.rarchives.ripme.utils.Http.response(Http.java:200)
	at com.rarchives.ripme.utils.Http.get(Http.java:159)
	at com.rarchives.ripme.ripper.rippers.XhamsterRipper.getFirstPage(XhamsterRipper.java:111)
	at com.rarchives.ripme.ripper.AbstractHTMLRipper.rip(AbstractHTMLRipper.java:94)
	at com.rarchives.ripme.ripper.AbstractRipper.run(AbstractRipper.java:616)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.SocketTimeoutException: Connect timed out
	at java.base/java.net.SocksSocketImpl.readSocksReply(SocksSocketImpl.java:138)
	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:509)
	at java.base/java.net.Socket.connect(Socket.java:609)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
	at java.base/sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:265)
	at java.base/sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:372)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:191)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
	at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:177)
	at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
	at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:449)
	at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:434)
	at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:181)
	at com.rarchives.ripme.utils.Http.response(Http.java:179)
	... 5 more

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in com.rarchives.ripme.utils.Http at the response, get, and failure paths named in the stack trace, then check how proxy.socks is passed into requests. Reproduce with the local SOCKS proxy at 127.0.0.1:9050 and a blocked URL such as the reported Xhamster URL. Done means the configured proxy is used and the URL loads successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
networking
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.