Azure / Azure/azure-rest-api-specs

[BUG] Put Blob from URL: Content-MD5 response header is absent despite documentation stating it is always returned

Open
#41,881 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug customer-reported data-plane question Service Attention Storage
Dominant language
TypeSpec
Stars
3.1k
Forks
6k
Avg merge
2d 21h
Merged PRs (30d)
432

Description

API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/storage/data-plane/Microsoft.BlobStorage/stable/2024-11-04/blob.json

API Spec version

2024-11-04

Describe the bug

The Put Blob from URL documentation states:

Content-MD5 — Returned for a block blob so that the client can check the integrity of message content. The Content-MD5 returned value is computed by Blob Storage. This header is returned even when the request doesn't include Content-MD5 or x-ms-blob-content-md5 headers.

In practice, when the source URL is a non-Azure HTTP endpoint that does not itself provide a Content-MD5 response header, the Content-MD5 header is absent from the Put Blob from URL response. Only x-ms-content-crc64 is returned.

However, Azure does silently compute and persist the MD5 as the x-ms-blob-content-md5 blob property — a subsequent Get Blob Properties call returns the correct hash. This storage-side behavior is not documented.

So in summary, there are two issues:

  1. Documentation inaccuracy: Content-MD5 is documented as always present in the Put Blob from URL response, but is absent when the source is a non-Azure HTTP URL that does not provide Content-MD5.
  2. Undocumented behavior: Azure computes and persists x-ms-blob-content-md5 as a blob property during the Put Blob from URL operation, which is not documented anywhere.
Expected behavior

When a block blob is created using the Put Blob from URL operation, the Content-MD5 response header should be present and contain the MD5 hash computed by Blob Storage, regardless of whether the source URL provides a Content-MD5 header. This is consistent with what the documentation states.

Actual behavior

When the source URL is a non-Azure HTTP endpoint that does not provide Content-MD5 in its response, the Content-MD5 header is absent from the Put Blob from URL response. Only x-ms-content-crc64 is returned.

However, Azure does compute the MD5 and persists it as the x-ms-blob-content-md5 blob property — a subsequent Get Blob Properties call returns the correct hash. This silent persistence is not documented.

Reproduction Steps
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using Azure.Storage.Blobs.Specialized;

// --- Configuration ---
// Use a publicly accessible URL that does NOT send Content-MD5 in its response.
var sourceUrl = new Uri(args.Length > 0 ? args[0] : "https://httpbin.org/bytes/1024");
var connectionString = args.Length > 1 ? args[1] : Environment.GetEnvironmentVariable("AZURE_STORAGE_CONNECTION_STRING")
    ?? throw new InvalidOperationException("Set AZURE_STORAGE_CONNECTION_STRING");

var containerName = $"md5-verify-{Guid.NewGuid():N}";
var containerClient = new BlobContainerClient(connectionString, containerName);
await containerClient.CreateIfNotExistsAsync();

try
{
    var blobName = $"test-{Guid.NewGuid():N}";
    var blockBlobClient = containerClient.GetBlockBlobClient(blobName);

    Console.WriteLine($"Source URL : {sourceUrl}");
    Console.WriteLine($"Blob       : {containerName}/{blobName}");
    Console.WriteLine();

    // 1. Upload from URL
    var options = new BlobSyncUploadFromUriOptions { CopySourceBlobProperties = true };
    var response = await blockBlobClient.SyncUploadFromUriAsync(sourceUrl, options);

    // 2. Check SDK typed property
    Console.WriteLine($"BlobContentInfo.ContentHash : {Format(response.Value.ContentHash)}");

    // 3. Check raw response headers
    var raw = response.GetRawResponse();
    Console.WriteLine();
    Console.WriteLine("Raw response headers (hash-related):");

    foreach (var header in raw.Headers)
    {
        if (header.Name.Contains("md5", StringComparison.OrdinalIgnoreCase) ||
            header.Name.Contains("crc", StringComparison.OrdinalIgnoreCase))
        {
            Console.WriteLine($"  {header.Name}: {header.Value}");
        }
    }

    // 4. Check stored blob property
    var props = await blockBlobClient.GetPropertiesAsync();
    Console.WriteLine();
    Console.WriteLine($"BlobProperties.ContentHash  : {Format(props.Value.ContentHash)}");

    // 5. Verdict
    Console.WriteLine();
    var sdkHash = response.Value.ContentHash;
    raw.Headers.TryGetValue("Content-MD5", out string? rawMd5);

    if (sdkHash is { Length: > 0 })
        Console.WriteLine("✓ SDK surfaces Content-MD5 — no bug.");
    else if (rawMd5 != null)
        Console.WriteLine("✗ REST API returns Content-MD5 but SDK does not surface it — file a bug.");
    else
        Console.WriteLine("✗ REST API does not return Content-MD5 — docs are inaccurate.");

    await blockBlobClient.DeleteIfExistsAsync();
}
finally
{
    await containerClient.DeleteIfExistsAsync();
}

static string Format(byte[]? hash) => hash is { Length: > 0 }
    ? Convert.ToBase64String(hash)
    : "(null or empty)";
Environment

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked specification/blob.json and locate the Put Blob from URL operation and its response headers. Run the supplied C# reproduction against a non-Azure source URL, then compare the response with a subsequent Get Blob Properties result. Done means the specification and related documentation accurately describe Content-MD5 and x-ms-blob-content-md5 behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
api, cloud, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.