googleapis / googleapis/google-api-dotnet-client

Manual chunk upload for GCS

Open
#2,480 12 comments 0 reactions 1 assignee Claimed by @rajeevpodar View on GitHub
priority: p3 type: feature request
Dominant language
C#
Stars
1.5k
Forks
558
Avg merge
16h 34m
Merged PRs (30d)
17

Description

Hello! Thank you for the work on this SDK as it has been all working perfectly for now.

We are trying to manually upload chunks the following way:

1. Client calls our backend service that, in turn, calls the APIs of `Google.Cloud.Storage.V1` to initialize a chunked upload of a large file (multiple GBs). For this we use ` client.CreateObjectUploader` and then `await uploader.InitiateSessionAsync`. We therefore get a session URI.

```
using MemoryStream emptyMemoryStream = new();
Google.Apis.Storage.v1.ObjectsResource.InsertMediaUpload uploader = client.CreateObjectUploader(command.BucketName, command.Key, GetMimeType(command.Key), emptyMemoryStream);

Uri uploadUri = await uploader.InitiateSessionAsync(cancellationToken);
```

2. Client manually upload chunks of data, 1 by 1, to our backend that, in turn, calls the APIs of `Google.Cloud.Storage.V1` .
Every time the client uploads a chunk of data, we do the following:

```
ResumableUpload actualUploader = ResumableUpload.CreateFromUploadUri(new Uri(request.ResumableUrl), new MemoryStream(Convert.FromBase64String(request.ContentAsBase64)));

IUploadProgress uploadProgress = await actualUploader.ResumeAsync(new Uri(request.ResumableUrl), cancellationToken);
```

Unfortunately, everytime a new chunk is uploaded, it replaces the one before that. We are not sure what we are missing as we have tried a few ways to do that.

Can you please let us know if you have any idea of what we are doing wrong?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.