googleapis / googleapis/google-cloud-cpp

Expose page token to caller for ListObjects API

Offen
#15,233 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
type: feature request
Vorherrschende Sprache
C++
Sterne
659
Forks
462
Ø Merge
1 T. 2 Std.
Gemergte PRs (30 T.)
89

Beschreibung

**What component of `google-cloud-cpp` is this feature request for?**

google/cloud/storage/client.h

**Is your feature request related to a problem? Please describe.**
Currently ListObjects returns an iterator which can be used to go through all the object metadata using single API call. But in case the process that was fetching crashed, I need to resume efficiently right from where it left off for a particular prefix. As in I don't want to start the iterator from the beginning for that prefix. If we can add functionality using page token(as other cloud provider like AWS exposes) to efficiently resume starting from the page where the process crashed, then that will prevent us from re-fetching from beginning.

**Describe the solution you'd like**

Some sort of pagination or efficient seeking mechanism to resume from the process left off.

**Describe alternatives you've considered**

I have not leveraged [startOffset] option but given the object's are listed in lexicographical order of their name, can it be used for efficiently resuming from where the process left off if I pass the last fetched key just before the crash as `startOffset` to a new ListObjects iterator ? Is this seek to the startOffset when passed in the ListObjects API done in O(1)?

**Additional context**

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start in google/cloud/storage/client.h at ListObjects and compare its iterator with the existing startOffset option described in the issue. Clarify the caller-facing pagination and resume contract, including whether startOffset provides the needed behavior, and define completion as a reliable way to continue listing after a crash.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
cpp
Bereich
api, cloud
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Muss geklärt werden
Anfängerfreundlichkeit
30/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.