googleapis / googleapis/google-cloud-python

google-cloud-pubsub SDK message pull is not working with SOCKS5 proxy

Abierto
#15,648 4 comentarios 0 reacciones 1 asignado Reclamado por @abbrowne126 Ver en GitHub
api: pubsub type: feature request
Lenguaje dominante
Python
Estrellas
5.4k
Forks
1.8k
Merge medio
3 d 4 h
PR fusionados (30 d)
122

Descripción

We are trying to build a pubsub integration to pull messages with proxy support using Pub/Sub SDK. It works fine with HTTP_PROXY but does not work with SOCKS5 proxy without internet connectivity.

#### Environment details

- OS type and version: Ubuntu 20.04.6 LTS (Focal Fossa)
- Python version: 3.8
- pip version: 3.8
- `google-cloud-pubsub` version: 2.17.1

#### Steps to reproduce
Use the mentioned code example to check the issue, but it is not working when the VM internet is off. The script works with HTTP_PROXY.
1. Create virtualenv
2. pip install google-cloud-pubsub
3. turn-off internet
4. Execute the script

#### Code example

```
import os
import json

from google.oauth2 import service_account
from google.api_core import retry
from google.cloud import pubsub_v1

import os

NO_PROXY = "localhost,127.0.0.1,0.0.0.0,localaddress"

# TODO
# require to create creds.json file on the same path with Google Service credentials JSON details
proxy_uri = "socks5h://:"
# proxy_uri = "http://:"
project_id = ""
subscription_id = ""

os.environ["no_proxy"] = NO_PROXY
os.environ["NO_PROXY"] = NO_PROXY
os.environ["http_proxy"] = proxy_uri
os.environ["HTTP_PROXY"] = proxy_uri
os.environ["https_proxy"] = proxy_uri
os.environ["HTTPS_PROXY"] = proxy_uri

def get_credentials():
dir_name = os.path.dirname(os.path.abspath(__file__))
creds_path = f"{dir_name}/creds.json"
credentials = service_account.Credentials.from_service_account_file(creds_path)
return credentials

def synchronous_pull(project_id, subscription_id):
subscriber = pubsub_v1.SubscriberClient(credentials=get_credentials())
subscription_path = subscriber.subscription_path(project_id, subscription_id)

NUM_MESSAGES = 10

with subscriber:
while(True):
print("Started receiving message...")
response = subscriber.pull(
request={"subscription": subscription_path, "max_messages": NUM_MESSAGES},
retry=retry.Retry(deadline=300),
)

if len(response.received_messages) == 0:
print(f"No messages to process.")
break

ack_ids = []
for received_message in response.received_messages:
print(f"Message: {received_message.message}")
ack_ids.append(received_message.ack_id)

subscriber.acknowledge(
request={"subscription": subscription_path, "ack_ids": ack_ids}
)

print(
f"Received and acknowledged {len(response.received_messages)} messages from {subscription_path}."
)

if __name__ == "__main__":
synchronous_pull(project_id, subscription_id)
```

#### Stack trace
```
E0630 08:03:56.145507323 2560525 http_proxy.cc:119] 'socks5h' scheme not supported in proxy URI
```

We tried to find the documents about the proxies supported by this SDK but had no luck.
Thanks!

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.