googleapis / googleapis/google-api-python-client

BrokenPipeError when using Attached Service Account on GCE Instance

Ouverte
#2,553 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
priority: p3 type: bug
Langage dominant
Python
Étoiles
8.9k
Forks
2.6k
Merge moyen
2 j 28 min
PR mergées (30 j)
17

Description

#### Environment details

- OS type and version: Google Compute Engine (GCE) with Container-Optimized OS
- Python version: `Python 3.7.16`
- pip version: `pip 21.3.1`
- `google-api-python-client`== 1.12.8
- `google-auth`==1.35.0
- `google-auth-httplib2`==0.2.0

#### Steps to reproduce

1. Deploy a Python application on a GCE instance with an attached service account.
2. Use `google.auth.default()` for authentication.
3. Let the service run for a few hours to allow the token to expire.
4. Perform a Google Drive API call during token refresh (e.g., listing files or creating a folder).

#### Code example

```python
gdrive_credentials: Credentials = (
Credentials.from_service_account_info(
gcp_credentials_dict,
scopes=SCOPES,
)
if gcp_credentials_dict
else google.auth.default(scopes=SCOPES)[0]
)

def create_services(credentials, scopes):
drive_service = build(
"drive", "v3", credentials=credentials
)
docs_service = build(
"docs", "v1", credentials=credentials
)

return credentials, drive_service, docs_service

def get_or_create_folder(self, drive_service, folder_name):
"""
Checks if a folder exists in the root directory and creates it if not.

:param drive_service: Authorized Drive API client.
:param folder_name: The name of the folder to check or create.
:return: The folder ID.
"""
# Search for the folder in the root directory
query = f"name = '{folder_name}' and mimeType = 'application/vnd.google-apps.folder' and 'root' in parents and trashed = false"
response = (
drive_service.files().list(q=query, fields="files(id, name)").execute()
)
files = response.get("files", [])

if files:
# Folder exists, return its ID
return files[0]["id"]
else:
# Folder doesn't exist, create it
folder_metadata = {
"name": folder_name,
"mimeType": "application/vnd.google-apps.folder",
"parents": ["root"], # Place in the root directory
}
folder = (
drive_service.files()
.create(body=folder_metadata, fields="id")
.execute()
)
return folder["id"]
```

#### Stacktrace

```bash
BrokenPipeError: [Errno 32] Broken pipe
File "/app/endpoints/webhooks.py", line 292, in get_services_status
current_app.config.get("BUILD_ENVIRONMENT"),
File "/app/middleware/gdocsprinter.py", line 43, in print_file_with_template
template_id, gdocs_drive_service, replace_template, build_environment
File "/app/middleware/gdocsprinter.py", line 111, in create_template_copy
drive_service, f"{build_environment}-prints"
File "/app/middleware/gdocsprinter.py", line 86, in get_or_create_folder
drive_service.files().list(q=query, fields="files(id, name)").execute()
File "googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "googleapiclient/http.py", line 909, in execute
headers=self.headers,
File "googleapiclient/http.py", line 204, in _retry_request
raise exception
File "googleapiclient/http.py", line 177, in _retry_request
resp, content = http.request(uri, method, *args, **kwargs)
File "google_auth_httplib2.py", line 190, in request
self._request, method, uri, request_headers)
File "/usr/local/lib/python3.7/site-packages/google/auth/credentials.py", line 133, in before_request
self.refresh(request)
File "/usr/local/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 111, in refresh
self._retrieve_info(request)
File "/usr/local/lib/python3.7/site-packages/google/auth/compute_engine/credentials.py", line 88, in _retrieve_info
request, service_account=self._service_account_email
File "/usr/local/lib/python3.7/site-packages/google/auth/compute_engine/_metadata.py", line 234, in get_service_account_info
return get(request, path, params={"recursive": "true"})
File "/usr/local/lib/python3.7/site-packages/google/auth/compute_engine/_metadata.py", line 150, in get
response = request(url=url, method="GET", headers=_METADATA_HEADERS)
File "google_auth_httplib2.py", line 117, in __call__
url, method=method, body=body, headers=headers, **kwargs)
File "__init__.py", line 1709, in request
conn, authority, uri, request_uri, method, body, headers, redirections, cachekey,
File "__init__.py", line 1424, in _request
(response, content) = self._conn_request(conn, request_uri, method, body, headers)
File "__init__.py", line 1347, in _conn_request
conn.request(method, request_uri, body, headers)
File "http/client.py", line 1281, in request
self._send_request(method, url, body, headers, encode_chunked)
File "http/client.py", line 1327, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "http/client.py", line 1276, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "http/client.py", line 1036, in _send_output
self.send(msg)
File "http/client.py", line 997, in send
self.sock.sendall(data)
```

#### Brief summary

After switching the authentication method for our application from service account keys to an attached service account on a Google Compute Engine (GCE) instance, we are encountering a BrokenPipeError during API calls to Google Drive.

The issue only occurs during token refresh after the application has been running for a couple of hours. Following the first failure, the service works again without intervention.

Thanks

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.