googleapis / googleapis/google-cloud-python

Documentation: Clarify that notification suffix filtering is not supported by GCS API

Abierto
#15,965 1 comentario 0 reacciones 1 asignado Reclamado por @chandra-siri Ver en GitHub
api: storage
Lenguaje dominante
Python
Estrellas
5.4k
Forks
1.8k
Merge medio
3 d 4 h
PR fusionados (30 d)
122

Descripción

### Environment details

- **OS**: Linux
- **Python version**: 3.11
- **google-cloud-storage version**: 3.4.0

### Summary

The `bucket.notification()` method does not support suffix filtering (e.g., `object_name_suffix` or `blob_name_suffix` parameters), which can lead to confusion when users attempt to create notifications that only trigger for files ending with specific suffixes like `.json` or `UploadCompleted.json`.

While this is a **limitation of the underlying GCS JSON API v1** (which only supports `object_name_prefix` in the [[notification resource schema](https://cloud.google.com/storage/docs/json_api/v1/notifications)](https://cloud.google.com/storage/docs/json_api/v1/notifications)), the Python client library documentation could be clearer about this limitation.

### Problem

Users attempting to filter GCS notifications by file suffix may try approaches like:

```python
notification = bucket.notification(
topic_name="my-topic",
blob_name_prefix="", # This works
# How do I filter by suffix? ❌ Not possible
)
```

The `bucket.notification()` signature shows only `blob_name_prefix` is available:

```python
def notification(
self,
topic_name=None,
topic_project=None,
custom_attributes=None,
event_types=None,
blob_name_prefix=None, # ← Only PREFIX supported
payload_format=NONE_PAYLOAD_FORMAT,
notification_id=None,
):
```

### Current Workarounds

Users must filter notifications in their application code:

```python
def process_notification(event):
# Application-level filtering since GCS doesn't support suffix
if not event.name.endswith("UploadCompleted.json"):
return # Skip non-sentinel files

# Process the notification
...
```

### Requested Changes

**Option 1: Add documentation note** (Minimal change)

Add a note to the `BucketNotification` class docstring and `bucket.notification()` method:

```python
def notification(...):
"""Factory: create a notification resource for the bucket.

Note: The GCS Notification API only supports prefix filtering via
blob_name_prefix. Suffix filtering (e.g., only notify for files
ending with '.json') is not supported by the GCS API. Applications
must implement suffix filtering in their notification handlers.

See: https://cloud.google.com/storage/docs/json_api/v1/notifications
"""
```

**Option 2: Add to samples** (Better)

Create a sample showing the recommended pattern for suffix-based filtering:

`samples/snippets/notification_with_suffix_filter.py`:
```python
"""Demonstrates application-level suffix filtering for GCS notifications.

Since GCS notifications only support prefix filtering, applications that
need to trigger only on specific file extensions must filter in code.
"""

def process_pubsub_notification(message):
"""Process Pub/Sub message with suffix filtering."""
import json
import base64

# Decode the GCS event
data = base64.b64decode(message.data).decode('utf-8')
event = json.loads(data)

object_name = event['name']

# Application-level suffix filtering
if not object_name.endswith('UploadCompleted.json'):
print(f"Skipping non-sentinel file: {object_name}")
message.ack()
return

# Process the sentinel file
print(f"Processing upload completion: {object_name}")
# ... your processing logic ...
message.ack()
```

### Why This Matters

This is a common pain point for users building event-driven pipelines. Many workflows need to trigger on specific sentinel files (like `UploadCompleted.json`, `_SUCCESS`, `manifest.json`) rather than every intermediate file upload.

Without clear documentation, users waste time trying to:
1. Find the "right" parameter name for suffix filtering
2. Debug why `_properties["objectNameSuffix"]` doesn't work
3. Upgrade library versions thinking it's a bug
4. File support tickets or Stack Overflow questions

### Related Issues

- googleapis/python-storage#1035 - Users requesting `matchGlob` for similar filtering needs
- This pattern appears in discussions about notification filtering across various forums

### Additional Context

The GCS REST API notification schema does not include `objectNameSuffix`:
- ✓ Supported: `object_name_prefix`
- ✗ Not supported: `object_name_suffix`

This appears to be an intentional design decision by the GCS team (prefix filtering is efficient with GCS's internal indexing, while suffix filtering would require scanning all object names).

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.