Line break in metadata item causes HMAC Failure
- Dominant language
- PowerShell
- Stars
- 597
- Forks
- 374
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 42
Description
Azure.Storage.Blobs version 12.24.0
I've been transferring 20 Million documents out of a SQL Server database into Azure Storage.
I had around 1000 Failures that logged an authentication failure below.
It turns out this was caused by a piece of meta data (x-ms-meta-filename) that had a line-break in it. Cleansing the data removing the line-break solved the problem.
The linebreak doesn't show in the headers below but it is definitely there. I would guess that the HMAC is including the linebreak in its calculation... but the data is being sent with the line break removed, which causes the HMAC validation failure.
x-ms-blob-type:BlockBlob
x-ms-client-request-id:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
x-ms-date:Fri, 18 Apr 2025 14:05:42 GMT
x-ms-meta-attachmenttype:ReferralFile
x-ms-meta-contenttype:application/rtf
x-ms-meta-filename:Gastroenterology (xxxxxxxx) - SPA.rtf
x-ms-meta-id:164348
x-ms-meta-sourceenvironment:BHR
x-ms-return-client-request-id:true
x-ms-version:2025-05-05
/xxxxxxxxxxxxxxxx/xxxxxxxxxxx-xxxxx-xxxx/xxxxxxxxxxxxxxxxxxx'.
Headers:
Server: Microsoft-HTTPAPI/2.0
x-ms-request-id: eaa0db57-101e-00ab-186a-b0f7c1000000
x-ms-error-code: AuthenticationFailed
Date: Fri, 18 Apr 2025 14:05:42 GMT
Content-Length: 1039
Content-Type: application/xml
at Azure.Storage.Blobs.BlockBlobRestClient.Upload(Int64 contentLength, Stream body, Nullable`1 timeout, Byte[] transactionalContentMD5, String blobContentType, String blobContentEncoding, String blobContentLanguage, Byte[] blobContentMD5, String blobCacheControl, IDictionary`2 metadata, String leaseId, String blobContentDisposition, String encryptionKey, String encryptionKeySha256, Nullable`1 encryptionAlgorithm, String encryptionScope, Nullable`1 tier, Nullable`1 ifModifiedSince, Nullable`1 ifUnmodifiedSince, String ifMatch, String ifNoneMatch, String ifTags, String blobTagsString, Nullable`1 immutabilityPolicyExpiry, Nullable`1 immutabilityPolicyMode, Nullable`1 legalHold, Byte[] transactionalContentCrc64, String structuredBodyType, Nullable`1 structuredContentLength, CancellationToken cancellationToken)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.UploadInternal(Stream content, BlobHttpHeaders blobHttpHeaders, IDictionary`2 metadata, IDictionary`2 tags, BlobRequestConditions conditions, Nullable`1 accessTier, BlobImmutabilityPolicy immutabilityPolicy, Nullable`1 legalHold, IProgress`1 progressHandler, UploadTransferValidationOptions transferValidationOverride, String operationName, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.Specialized.BlockBlobClient.<>c__DisplayClass65_0.<b__0>d.MoveNext()
--- End of stack trace from previous location ---
at Azure.Storage.PartitionedUploader`2.UploadInternal(Stream content, Nullable`1 expectedContentLength, TServiceSpecificData args, IProgress`1 progressHandler, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.BlobClient.StagedUploadInternal(Stream content, BlobUploadOptions options, Boolean async, CancellationToken cancellationToken)
at Azure.Storage.Blobs.BlobClient.Upload(Stream content, BlobUploadOptions options, CancellationToken cancellationToken)
at AzureStorageProxies.AzureBlobStorageUploader.UploadFile(MemoryStream memoryStream, String blobName, IDictionary`2 metaData, String contentType, CancellationToken cancellationToken) in C:\Users\xxxxxxx\dev-med\xxxxxx.AzureDataMigration\AzureStorageProxies\AzureBlobStorageUploader.cs:line 78
2025-04-18 14:05:43 [ERR] [Thread:] Error uploading file stream as '5f73356efaed452c9f11a87d1ac7ad8b': Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:eaa0db57-101e-00ab-186a-b0f7c1000000
Time:2025-04-18T14:05:42.9988773Z
Status: 403 (Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.)
ErrorCode: AuthenticationFailed
Additional Information:
AuthenticationErrorDetail: The MAC signature found in the HTTP request 'G9Ofp/ne2t6XawxPhtZfiKA1QZ6jC/AF10c1bSa32N4=' is not the same as any computed signature. Server used following string to sign: 'PUT
1667420
application/octet-stream
Contributor guide
Research direction
Start with Azure.Storage.Blobs 12.24.0 and the BlockBlobClient.UploadInternal stack entry; reproduce an upload whose x-ms-meta-filename contains a line break and compare the sent metadata with the server's string-to-sign. Done means the request no longer produces AuthenticationFailed for this metadata case, with a regression check for the upload path.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp
- Domain
- cloud, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100