microsoftgraph / microsoftgraph/msgraph-sdk-dotnet-core

Support `deferCommit` in `LargeFileUploadTask` for upload sessions

Open
#984 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type:feature
Dominant language
C#
Stars
164
Forks
65
Avg merge
38m
Merged PRs (30d)
1

Description

I want to defer the final creation of the file being uploaded via an upload session in the destination (in this case, SharePoint Embedded) until I explicitly make a request to complete the upload. For this, I'm setting the deferCommit property in the request arguments for the creation of the upload session, as indicated in Create an upload session - driveItem: createUploadSession | MS Learn.

When using LargeFileUploadTask to use the created upload session, as indicated in Upload large files using the Microsoft Graph SDKs | MS Learn, the upload task does not stop when all the bytes have been uploaded and the server returns that there are no nextExpectedRanges in the 202 response. It starts a loop to retrieve the status of the upload session and the expected ranges (which in return are empty) until all retry attempts have been exhausted. Then, it throws an exception saying Upload failed too many times..

From source: LargeFileUploadTask.cs

public async Task<UploadResult<T>> UploadAsync(IProgress<long> progress = null, int maxTries = 3, CancellationToken cancellationToken = default)
{
    var uploadTries = 0;
    var trackedExceptions = new List<Exception>();

    while (uploadTries < maxTries)
    {
        var sliceRequests = this.GetUploadSliceRequests();

        foreach (var request in sliceRequests)
        {
            var uploadResult = await this.UploadSliceAsync(request, trackedExceptions, cancellationToken).ConfigureAwait(false);

            progress?.Report(request.RangeEnd);//report the progress of upload (how many bytes have been uploaded so far)

            if (uploadResult.UploadSucceeded) // **<--- from observation/testing: this is always false**
            {
                return uploadResult;
            }

            ThrowIfUploadCancelled(trackedExceptions, cancellationToken);
        }

        await this.UpdateSessionStatusAsync(cancellationToken).ConfigureAwait(false);
        uploadTries += 1;
        if (uploadTries < maxTries)
        {
            // Exponential back off in case of failures.
            await Task.Delay(2000 * uploadTries * uploadTries, cancellationToken).ConfigureAwait(false);
        }

        ThrowIfUploadCancelled(trackedExceptions, cancellationToken);
    }

    throw new TaskCanceledException("Upload failed too many times. See InnerException for list of exceptions that occured.", new AggregateException(trackedExceptions.ToArray()));
}

public async Task<IUploadSession> UpdateSessionStatusAsync(CancellationToken cancellationToken = default)
{
    var requestBuilder = new UploadSessionRequestBuilder(this.Session, this._requestAdapter);
    var newSession = await requestBuilder.GetAsync(cancellationToken).ConfigureAwait(false);

    var newRangesRemaining = this.GetRangesRemaining(newSession);  // **<--- from observation/testing: this return empty**

    this._rangesRemaining = newRangesRemaining;
    newSession.UploadUrl = this.Session.UploadUrl; // Sometimes the UploadUrl is not returned
    this.Session = newSession;
    return newSession;
}
Proposed solution

I would like LargeFileUploadTask to handle defer commits. Alternatives:

  1. Allow the caller to indicate that the upload is deferred during construction (i.e.: a parameter).
  2. The task itself can interpret the 202 response with no next expected ranges as having the commit deferred.

Once the task is done uploading, there could be a CommitSessionAsync method (in the same line as the DeleteSessionAsync method) to perform the commit request.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/Microsoft.Graph.Core/Tasks/LargeFileUploadTask.cs, especially UploadAsync, UpdateSessionStatusAsync, and the existing DeleteSessionAsync method. Read the linked Microsoft Learn upload-session documentation and determine how deferred commits should be represented. Done means deferred uploads stop after all bytes are sent and expose an explicit way to complete the session without retry exhaustion.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.