Azure / Azure/Azurite

Batch operation for Table service doesn't work with production urls and Microsoft.Azure.Cosmos.Table nuget

Open
#1,141 0 comments 1 reaction 0 assignees View on GitHub
bug investigation table-storage
Dominant language
TypeScript
Stars
2.3k
Forks
393
Avg merge
1d 20h
Merged PRs (30d)
36

Description

### Which service(blob, file, queue, table) does this issue concern?
table
### Which version of the Azurite was used?
3.14.3
### Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
npm
### What's the Node.js version?
14.17.0
### What problem was encountered?
Inserting a TableEntity in a table using a batch operation fails. Inserting a TableEntity as a single operation works.
### Steps to reproduce the issue?
Set up an account (in this example testaccount) in azurite using production url according to documentation with environment variable and hosts file
```
127.0.0.1 testaccount.blob.tibp.int
127.0.0.1 testaccount.queue.tibp.int
127.0.0.1 testaccount.table.tibp.int
```
Run the following code in a .net core 3.1 console app with Microsoft.Azure.Cosmos.Table 1.0.8 nuget
```
var connectionString = "DefaultEndpointsProtocol=http;AccountName=testaccount;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://testaccount.blob.tibp.int:10000;QueueEndpoint=http://testaccount.queue.tibp.int:10001;TableEndpoint=http://testaccount.table.tibp.int:10002;";
var cloudStorageAccount = CloudStorageAccount.Parse(connectionString);
var client = cloudStorageAccount.CreateCloudTableClient();

var table = client.GetTableReference("table1");
await table.CreateIfNotExistsAsync();
await table.ExecuteAsync(TableOperation.InsertOrReplace(new TableEntity {PartitionKey = "Partition1", RowKey = "Row1"}));
var batch = new TableBatchOperation
{
TableOperation.InsertOrReplace(new TableEntity {PartitionKey = "Partition1", RowKey = "Row2"})
};
await table.ExecuteBatchAsync(batch);
```
The last row fails but all other operations succeeds, i.e. the table is created and contains one row with row key Row1.
Log for the successful insert:
```
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableStorageContextMiddleware: RequestMethod=PUT RequestURL=http://testaccount.table.tibp.int/table1(PartitionKey='Partition1',RowKey='Row1') RequestHeaders:{"host":"testaccount.table.tibp.int:10002","accept-charset":"UTF-8","maxdataserviceversion":"3.0;NetFx","accept":"application/json; odata=minimalmetadata","dataserviceversion":"3.0;","x-ms-client-request-id":"07d64c08-1d1e-4f3b-aac1-46f4f6c814d8","user-agent":"Azure-Cosmos-Table/1.0.8 (.NET CLR 3.1.20; Win32NT 6.2.9200.0)","x-ms-version":"2017-07-29","x-ms-date":"Sat, 30 Oct 2021 12:43:26 GMT","authorization":"SharedKey testaccount:grDE3K2fkCU2ZF4rSG0qrCGVIOTGofA3vTS1At7AxfY=","content-type":"application/json","content-length":"2"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 debug: tableStorageContextMiddleware: Dispatch pattern string: /table1(PartitionKey='PLACEHOLDER',RowKey='PLACEHOLDER')
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: tableStorageContextMiddleware: Account=testaccount tableName=table1
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: DispatchMiddleware: Dispatching request...
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: DispatchMiddleware: Operation=Table_UpdateEntity
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyLiteAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyLiteAuthenticator:validate() Request doesn't include valid authentication header. Skip SharedKeyLite authentication.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\napplication/json\nSat, 30 Oct 2021 12:43:26 GMT\n/testaccount/table1(PartitionKey='Partition1',RowKey='Row1')"
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey testaccount:grDE3K2fkCU2ZF4rSG0qrCGVIOTGofA3vTS1At7AxfY=
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() Signature 1 matched.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: DeserializerMiddleware: Start deserializing...
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 debug: deserialize(): Raw request body string is (removed all empty characters) {}
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: HandlerMiddleware: DeserializedParameters={"options":{"queryOptions":{},"requestId":"07d64c08-1d1e-4f3b-aac1-46f4f6c814d8","dataServiceVersion":"3.0;","tableEntityProperties":{}},"version":"2017-07-29","body":"ReadableStream"}
2021-10-30T12:43:26.235Z warn: TableHandler:updateEntity() Incoming PartitionKey:Partition1 RowKey:Row1 in URL parameters don't align with entity body PartitionKey:undefined RowKey:undefined.
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: SerializerMiddleware: Start serializing...
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: EndMiddleware: End response. TotalTimeInMS=2 StatusCode=204 StatusMessage=undefined Headers={"server":"Azurite-Table/3.14.3","x-ms-client-request-id":"07d64c08-1d1e-4f3b-aac1-46f4f6c814d8","x-ms-request-id":"a40bb2d6-c143-4978-b816-10e4dd967c13","x-ms-version":"2020-10-02","date":"Sat, 30 Oct 2021 12:43:26 GMT","etag":"W/\"datetime'2021-10-30T12%3A43%3A26.2330000Z'\""}
```
Log for the failing batch request:
```
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableStorageContextMiddleware: RequestMethod=POST RequestURL=http://testaccount.table.tibp.int/$batch RequestHeaders:{"host":"testaccount.table.tibp.int:10002","accept-charset":"UTF-8","maxdataserviceversion":"3.0;NetFx","accept":"application/json; odata=minimalmetadata","dataserviceversion":"3.0;","x-ms-client-request-id":"a9623db2-17be-46cc-8dc2-dda30b0bdee7","user-agent":"Azure-Cosmos-Table/1.0.8 (.NET CLR 3.1.20; Win32NT 6.2.9200.0)","x-ms-version":"2017-07-29","x-ms-date":"Sat, 30 Oct 2021 12:44:39 GMT","authorization":"SharedKey testaccount:mtibvkoyBfaUZon4Hi4okrA2Axyv+s23qof3fJQLc9U=","content-type":"multipart/mixed; boundary=batch_a68af5c4-e290-4bd9-be51-1687e038e0a4","content-length":"579"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb debug: tableStorageContextMiddleware: Dispatch pattern string: /$batch
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: tableStorageContextMiddleware: Account=testaccount tableName=$batch
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb verbose: DispatchMiddleware: Dispatching request...
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: DispatchMiddleware: Operation=Table_Batch
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyLiteAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyLiteAuthenticator:validate() Request doesn't include valid authentication header. Skip SharedKeyLite authentication.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() [STRING TO SIGN]:"POST\n\nmultipart/mixed; boundary=batch_a68af5c4-e290-4bd9-be51-1687e038e0a4\nSat, 30 Oct 2021 12:44:39 GMT\n/testaccount/$batch"
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey testaccount:mtibvkoyBfaUZon4Hi4okrA2Axyv+s23qof3fJQLc9U=
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() Signature 1 matched.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb verbose: DeserializerMiddleware: Start deserializing...
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: HandlerMiddleware: DeserializedParameters={"version":"2017-07-29","options":{"requestId":"a9623db2-17be-46cc-8dc2-dda30b0bdee7","dataServiceVersion":"3.0;"},"multipartContentType":"multipart/mixed; boundary=batch_a68af5c4-e290-4bd9-be51-1687e038e0a4","contentLength":579,"body":"ReadableStream"}
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb debug: TableHandler:batch() Raw request string is "--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4\r\nContent-Type: multipart/mixed; boundary=changeset_84de5d9b-4978-431f-a447-d6e9405fa332\r\n\r\n--changeset_84de5d9b-4978-431f-a447-d6e9405fa332\r\nContent-Type: application/http\r\nContent-Transfer-Encoding: binary\r\n\r\nPUT http://testaccount.table.tibp.int:10002/table1(PartitionKey='Partition1',RowKey='Row2') HTTP/1.1\r\nAccept: application/json;odata=minimalmetadata\r\nContent-Type: application/json\r\nDataServiceVersion: 3.0;\r\n\r\n{}\r\n--changeset_84de5d9b-4978-431f-a447-d6e9405fa332--\r\n--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4--\r\n"
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb error: ErrorMiddleware: Received an error, fill error information to HTTP response
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb error: ErrorMiddleware: ErrorName=Error ErrorMessage=Couldn't extract path from URL in sub-Request:

PUT http://testaccount.table.tibp.int:10002/table1(PartitionKey='Partition1',RowKey='Row2') HTTP/1.1
Accept: application/json;odata=minimalmetadata
Content-Type: application/json
DataServiceVersion: 3.0;

{}
--changeset_84de5d9b-4978-431f-a447-d6e9405fa332--
--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4--
ErrorStack="Error: Couldn't extract path from URL in sub-Request:\n \r\n\r\nPUT http://testaccount.table.tibp.int:10002/table1(PartitionKey='Partition1',RowKey='Row2') HTTP/1.1\r\nAccept: application/json;odata=minimalmetadata\r\nContent-Type: application/json\r\nDataServiceVersion: 3.0;\r\n\r\n{}\r\n--changeset_84de5d9b-4978-431f-a447-d6e9405fa332--\r\n--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4--\r\n\n at C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\batch\\TableBatchSerialization.js:70:23\n at Array.map ()\n at TableBatchSerialization.deserializeBatchRequest (C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\batch\\TableBatchSerialization.js:57:45)\n at TableBatchOrchestrator.processBatchRequestAndSerializeResponse (C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\batch\\TableBatchOrchestrator.js:44:32)\n at TableHandler.batch (C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\handlers\\TableHandler.js:466:50)\n at runMicrotasks ()\n at processTicksAndRejections (internal/process/task_queues.js:95:5)"
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb error: ErrorMiddleware: Set HTTP code: 500
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: EndMiddleware: End response. TotalTimeInMS=1 StatusCode=500 StatusMessage=undefined Headers={"server":"Azurite-Table/3.14.3"}
```

### Have you found a mitigation/solution?
No. The only workaround is to only use single operations.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.