googleapis / googleapis/google-cloud-python
google-cloud-pubsub SDK message pull is not working with SOCKS5 proxy
- Langage dominant
- Python
- Étoiles
- 5.4k
- Forks
- 1.8k
- Merge moyen
- 3 j 4 h
- PR mergées (30 j)
- 122
Description
We are trying to build a pubsub integration to pull messages with proxy support using Pub/Sub SDK. It works fine with HTTP_PROXY but does not work with SOCKS5 proxy without internet connectivity.
#### Environment details
- OS type and version: Ubuntu 20.04.6 LTS (Focal Fossa)
- Python version: 3.8
- pip version: 3.8
- `google-cloud-pubsub` version: 2.17.1
#### Steps to reproduce
Use the mentioned code example to check the issue, but it is not working when the VM internet is off. The script works with HTTP_PROXY.
1. Create virtualenv
2. pip install google-cloud-pubsub
3. turn-off internet
4. Execute the script
#### Code example
```
import os
import json
from google.oauth2 import service_account
from google.api_core import retry
from google.cloud import pubsub_v1
import os
NO_PROXY = "localhost,127.0.0.1,0.0.0.0,localaddress"
# TODO
# require to create creds.json file on the same path with Google Service credentials JSON details
proxy_uri = "socks5h://:"
# proxy_uri = "http://:"
project_id = ""
subscription_id = ""
os.environ["no_proxy"] = NO_PROXY
os.environ["NO_PROXY"] = NO_PROXY
os.environ["http_proxy"] = proxy_uri
os.environ["HTTP_PROXY"] = proxy_uri
os.environ["https_proxy"] = proxy_uri
os.environ["HTTPS_PROXY"] = proxy_uri
def get_credentials():
dir_name = os.path.dirname(os.path.abspath(__file__))
creds_path = f"{dir_name}/creds.json"
credentials = service_account.Credentials.from_service_account_file(creds_path)
return credentials
def synchronous_pull(project_id, subscription_id):
subscriber = pubsub_v1.SubscriberClient(credentials=get_credentials())
subscription_path = subscriber.subscription_path(project_id, subscription_id)
NUM_MESSAGES = 10
with subscriber:
while(True):
print("Started receiving message...")
response = subscriber.pull(
request={"subscription": subscription_path, "max_messages": NUM_MESSAGES},
retry=retry.Retry(deadline=300),
)
if len(response.received_messages) == 0:
print(f"No messages to process.")
break
ack_ids = []
for received_message in response.received_messages:
print(f"Message: {received_message.message}")
ack_ids.append(received_message.ack_id)
subscriber.acknowledge(
request={"subscription": subscription_path, "ack_ids": ack_ids}
)
print(
f"Received and acknowledged {len(response.received_messages)} messages from {subscription_path}."
)
if __name__ == "__main__":
synchronous_pull(project_id, subscription_id)
```
#### Stack trace
```
E0630 08:03:56.145507323 2560525 http_proxy.cc:119] 'socks5h' scheme not supported in proxy URI
```
We tried to find the documents about the proxies supported by this SDK but had no luck.
Thanks!
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.