googleapis / googleapis/google-cloud-python
Documentation: Clarify that notification suffix filtering is not supported by GCS API
- Vorherrschende Sprache
- Python
- Sterne
- 5.4k
- Forks
- 1.8k
- Ø Merge
- 3 T. 4 Std.
- Gemergte PRs (30 T.)
- 122
Beschreibung
### Environment details
- **OS**: Linux
- **Python version**: 3.11
- **google-cloud-storage version**: 3.4.0
### Summary
The `bucket.notification()` method does not support suffix filtering (e.g., `object_name_suffix` or `blob_name_suffix` parameters), which can lead to confusion when users attempt to create notifications that only trigger for files ending with specific suffixes like `.json` or `UploadCompleted.json`.
While this is a **limitation of the underlying GCS JSON API v1** (which only supports `object_name_prefix` in the [[notification resource schema](https://cloud.google.com/storage/docs/json_api/v1/notifications)](https://cloud.google.com/storage/docs/json_api/v1/notifications)), the Python client library documentation could be clearer about this limitation.
### Problem
Users attempting to filter GCS notifications by file suffix may try approaches like:
```python
notification = bucket.notification(
topic_name="my-topic",
blob_name_prefix="", # This works
# How do I filter by suffix? ❌ Not possible
)
```
The `bucket.notification()` signature shows only `blob_name_prefix` is available:
```python
def notification(
self,
topic_name=None,
topic_project=None,
custom_attributes=None,
event_types=None,
blob_name_prefix=None, # ← Only PREFIX supported
payload_format=NONE_PAYLOAD_FORMAT,
notification_id=None,
):
```
### Current Workarounds
Users must filter notifications in their application code:
```python
def process_notification(event):
# Application-level filtering since GCS doesn't support suffix
if not event.name.endswith("UploadCompleted.json"):
return # Skip non-sentinel files
# Process the notification
...
```
### Requested Changes
**Option 1: Add documentation note** (Minimal change)
Add a note to the `BucketNotification` class docstring and `bucket.notification()` method:
```python
def notification(...):
"""Factory: create a notification resource for the bucket.
Note: The GCS Notification API only supports prefix filtering via
blob_name_prefix. Suffix filtering (e.g., only notify for files
ending with '.json') is not supported by the GCS API. Applications
must implement suffix filtering in their notification handlers.
See: https://cloud.google.com/storage/docs/json_api/v1/notifications
"""
```
**Option 2: Add to samples** (Better)
Create a sample showing the recommended pattern for suffix-based filtering:
`samples/snippets/notification_with_suffix_filter.py`:
```python
"""Demonstrates application-level suffix filtering for GCS notifications.
Since GCS notifications only support prefix filtering, applications that
need to trigger only on specific file extensions must filter in code.
"""
def process_pubsub_notification(message):
"""Process Pub/Sub message with suffix filtering."""
import json
import base64
# Decode the GCS event
data = base64.b64decode(message.data).decode('utf-8')
event = json.loads(data)
object_name = event['name']
# Application-level suffix filtering
if not object_name.endswith('UploadCompleted.json'):
print(f"Skipping non-sentinel file: {object_name}")
message.ack()
return
# Process the sentinel file
print(f"Processing upload completion: {object_name}")
# ... your processing logic ...
message.ack()
```
### Why This Matters
This is a common pain point for users building event-driven pipelines. Many workflows need to trigger on specific sentinel files (like `UploadCompleted.json`, `_SUCCESS`, `manifest.json`) rather than every intermediate file upload.
Without clear documentation, users waste time trying to:
1. Find the "right" parameter name for suffix filtering
2. Debug why `_properties["objectNameSuffix"]` doesn't work
3. Upgrade library versions thinking it's a bug
4. File support tickets or Stack Overflow questions
### Related Issues
- googleapis/python-storage#1035 - Users requesting `matchGlob` for similar filtering needs
- This pattern appears in discussions about notification filtering across various forums
### Additional Context
The GCS REST API notification schema does not include `objectNameSuffix`:
- ✓ Supported: `object_name_prefix`
- ✗ Not supported: `object_name_suffix`
This appears to be an intentional design decision by the GCS team (prefix filtering is efficient with GCS's internal indexing, while suffix filtering would require scanning all object names).
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.