Azure / Azure/azure-sdk

Line break in metadata item causes HMAC Failure

Open
#8,752 0 comments 0 reactions 0 assignees View on GitHub
Client customer-reported needs-team-triage question Storage
Dominant language
PowerShell
Stars
597
Forks
374
Avg merge
5d 21h
Merged PRs (30d)
42

Description

Azure.Storage.Blobs version 12.24.0

I've been transferring 20 Million documents out of a SQL Server database into Azure Storage.
I had around 1000 Failures that logged an authentication failure below.
It turns out this was caused by a piece of meta data (x-ms-meta-filename) that had a line-break in it. Cleansing the data removing the line-break solved the problem.

The linebreak doesn't show in the headers below but it is definitely there. I would guess that the HMAC is including the linebreak in its calculation... but the data is being sent with the line break removed, which causes the HMAC validation failure.

x-ms-blob-type:BlockBlob
x-ms-client-request-id:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
x-ms-date:Fri, 18 Apr 2025 14:05:42 GMT
x-ms-meta-attachmenttype:ReferralFile
x-ms-meta-contenttype:application/rtf
x-ms-meta-filename:Gastroenterology (xxxxxxxx) - SPA.rtf
x-ms-meta-id:164348
x-ms-meta-sourceenvironment:BHR
x-ms-return-client-request-id:true
x-ms-version:2025-05-05
/xxxxxxxxxxxxxxxx/xxxxxxxxxxx-xxxxx-xxxx/xxxxxxxxxxxxxxxxxxx'.

Headers:
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: eaa0db57-101e-00ab-186a-b0f7c1000000
x-ms-error-code: AuthenticationFailed
Date: Fri, 18 Apr 2025 14:05:42 GMT
Content-Length: 1039
Content-Type: application/xml

at Azure.Storage.Blobs.BlockBlobRestClient.Upload(Int64 contentLength, Stream body, Nullable`1 timeout, Byte[] transactionalContentMD5, String blobContentType, String blobContentEncoding, String blobContentLanguage, Byte[] blobContentMD5, String blobCacheControl, IDictionary`2 metadata, String leaseId, String blobContentDisposition, String encryptionKey, String encryptionKeySha256, Nullable`1 encryptionAlgorithm, String encryptionScope, Nullable`1 tier, Nullable`1 ifModifiedSince, Nullable`1 ifUnmodifiedSince, String ifMatch, String ifNoneMatch, String ifTags, String blobTagsString, Nullable`1 immutabilityPolicyExpiry, Nullable`1 immutabilityPolicyMode, Nullable`1 legalHold, Byte[] transactionalContentCrc64, String structuredBodyType, Nullable`1 structuredContentLength, CancellationToken cancellationToken)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadInternal(Stream content, BlobHttpHeaders blobHttpHeaders, IDictionary`2 metadata, IDictionary`2 tags, BlobRequestConditions conditions, Nullable`1 accessTier, BlobImmutabilityPolicy immutabilityPolicy, Nullable`1 legalHold, IProgress`1 progressHandler, UploadTransferValidationOptions transferValidationOverride, String operationName, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.<>c__DisplayClass65_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Azure.Storage.PartitionedUploader`2.UploadInternal(Stream content, Nullable`1 expectedContentLength, TServiceSpecificData args, IProgress`1 progressHandler, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.BlobClient.StagedUploadInternal(Stream content, BlobUploadOptions options, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.BlobClient.Upload(Stream content, BlobUploadOptions options, CancellationToken cancellationToken)
at AzureStorageProxies.AzureBlobStorageUploader.UploadFile(MemoryStream memoryStream, String blobName, IDictionary`2 metaData, String contentType, CancellationToken cancellationToken) in C:\Users\xxxxxxx\dev-med\xxxxxx.AzureDataMigration\AzureStorageProxies\AzureBlobStorageUploader.cs:line 78
2025-04-18 14:05:43 [ERR] [Thread:] Error uploading file stream as '5f73356efaed452c9f11a87d1ac7ad8b': Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:eaa0db57-101e-00ab-186a-b0f7c1000000
Time:2025-04-18T14:05:42.9988773Z
Status: 403 (Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.)
ErrorCode: AuthenticationFailed

Additional Information:
AuthenticationErrorDetail: The MAC signature found in the HTTP request 'G9Ofp/ne2t6XawxPhtZfiKA1QZ6jC/AF10c1bSa32N4=' is not the same as any computed signature. Server used following string to sign: 'PUT

1667420

application/octet-stream

Contributor guide

Open the contributing guide

Research direction

Start with Azure.Storage.Blobs 12.24.0 and the BlockBlobClient.UploadInternal stack entry; reproduce an upload whose x-ms-meta-filename contains a line break and compare the sent metadata with the server's string-to-sign. Done means the request no longer produces AuthenticationFailed for this metadata case, with a regression check for the upload path.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
cloud, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.