googleapis / googleapis/google-cloud-python

google-cloud-pubsub SDK message pull is not working with SOCKS5 proxy

Aperta
#15,648 4 commenti 0 reazioni 1 assegnatario Rivendicata da @abbrowne126 Vedi su GitHub
api: pubsub type: feature request
Lingua principale
Python
Stelle
5.4k
Fork
1.8k
Merge medio
3g 4h
PR unite (30g)
122

Descrizione

We are trying to build a pubsub integration to pull messages with proxy support using Pub/Sub SDK. It works fine with HTTP_PROXY but does not work with SOCKS5 proxy without internet connectivity.

#### Environment details

- OS type and version: Ubuntu 20.04.6 LTS (Focal Fossa)
- Python version: 3.8
- pip version: 3.8
- `google-cloud-pubsub` version: 2.17.1

#### Steps to reproduce
Use the mentioned code example to check the issue, but it is not working when the VM internet is off. The script works with HTTP_PROXY.
1. Create virtualenv
2. pip install google-cloud-pubsub
3. turn-off internet
4. Execute the script

#### Code example

```
import os
import json

from google.oauth2 import service_account
from google.api_core import retry
from google.cloud import pubsub_v1

import os

NO_PROXY = "localhost,127.0.0.1,0.0.0.0,localaddress"

# TODO
# require to create creds.json file on the same path with Google Service credentials JSON details
proxy_uri = "socks5h://:"
# proxy_uri = "http://:"
project_id = ""
subscription_id = ""

os.environ["no_proxy"] = NO_PROXY
os.environ["NO_PROXY"] = NO_PROXY
os.environ["http_proxy"] = proxy_uri
os.environ["HTTP_PROXY"] = proxy_uri
os.environ["https_proxy"] = proxy_uri
os.environ["HTTPS_PROXY"] = proxy_uri

def get_credentials():
dir_name = os.path.dirname(os.path.abspath(__file__))
creds_path = f"{dir_name}/creds.json"
credentials = service_account.Credentials.from_service_account_file(creds_path)
return credentials

def synchronous_pull(project_id, subscription_id):
subscriber = pubsub_v1.SubscriberClient(credentials=get_credentials())
subscription_path = subscriber.subscription_path(project_id, subscription_id)

NUM_MESSAGES = 10

with subscriber:
while(True):
print("Started receiving message...")
response = subscriber.pull(
request={"subscription": subscription_path, "max_messages": NUM_MESSAGES},
retry=retry.Retry(deadline=300),
)

if len(response.received_messages) == 0:
print(f"No messages to process.")
break

ack_ids = []
for received_message in response.received_messages:
print(f"Message: {received_message.message}")
ack_ids.append(received_message.ack_id)

subscriber.acknowledge(
request={"subscription": subscription_path, "ack_ids": ack_ids}
)

print(
f"Received and acknowledged {len(response.received_messages)} messages from {subscription_path}."
)

if __name__ == "__main__":
synchronous_pull(project_id, subscription_id)
```

#### Stack trace
```
E0630 08:03:56.145507323 2560525 http_proxy.cc:119] 'socks5h' scheme not supported in proxy URI
```

We tried to find the documents about the proxies supported by this SDK but had no luck.
Thanks!

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.