Batch operation for Table service doesn't work with production urls and Microsoft.Azure.Cosmos.Table nuget
- Dominant language
- TypeScript
- Stars
- 2.3k
- Forks
- 393
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 36
Description
### Which service(blob, file, queue, table) does this issue concern?
table
### Which version of the Azurite was used?
3.14.3
### Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
npm
### What's the Node.js version?
14.17.0
### What problem was encountered?
Inserting a TableEntity in a table using a batch operation fails. Inserting a TableEntity as a single operation works.
### Steps to reproduce the issue?
Set up an account (in this example testaccount) in azurite using production url according to documentation with environment variable and hosts file
```
127.0.0.1 testaccount.blob.tibp.int
127.0.0.1 testaccount.queue.tibp.int
127.0.0.1 testaccount.table.tibp.int
```
Run the following code in a .net core 3.1 console app with Microsoft.Azure.Cosmos.Table 1.0.8 nuget
```
var connectionString = "DefaultEndpointsProtocol=http;AccountName=testaccount;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://testaccount.blob.tibp.int:10000;QueueEndpoint=http://testaccount.queue.tibp.int:10001;TableEndpoint=http://testaccount.table.tibp.int:10002;";
var cloudStorageAccount = CloudStorageAccount.Parse(connectionString);
var client = cloudStorageAccount.CreateCloudTableClient();
var table = client.GetTableReference("table1");
await table.CreateIfNotExistsAsync();
await table.ExecuteAsync(TableOperation.InsertOrReplace(new TableEntity {PartitionKey = "Partition1", RowKey = "Row1"}));
var batch = new TableBatchOperation
{
TableOperation.InsertOrReplace(new TableEntity {PartitionKey = "Partition1", RowKey = "Row2"})
};
await table.ExecuteBatchAsync(batch);
```
The last row fails but all other operations succeeds, i.e. the table is created and contains one row with row key Row1.
Log for the successful insert:
```
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableStorageContextMiddleware: RequestMethod=PUT RequestURL=http://testaccount.table.tibp.int/table1(PartitionKey='Partition1',RowKey='Row1') RequestHeaders:{"host":"testaccount.table.tibp.int:10002","accept-charset":"UTF-8","maxdataserviceversion":"3.0;NetFx","accept":"application/json; odata=minimalmetadata","dataserviceversion":"3.0;","x-ms-client-request-id":"07d64c08-1d1e-4f3b-aac1-46f4f6c814d8","user-agent":"Azure-Cosmos-Table/1.0.8 (.NET CLR 3.1.20; Win32NT 6.2.9200.0)","x-ms-version":"2017-07-29","x-ms-date":"Sat, 30 Oct 2021 12:43:26 GMT","authorization":"SharedKey testaccount:grDE3K2fkCU2ZF4rSG0qrCGVIOTGofA3vTS1At7AxfY=","content-type":"application/json","content-length":"2"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 debug: tableStorageContextMiddleware: Dispatch pattern string: /table1(PartitionKey='PLACEHOLDER',RowKey='PLACEHOLDER')
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: tableStorageContextMiddleware: Account=testaccount tableName=table1
2021-10-30T12:43:26.233Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: DispatchMiddleware: Dispatching request...
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: DispatchMiddleware: Operation=Table_UpdateEntity
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyLiteAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyLiteAuthenticator:validate() Request doesn't include valid authentication header. Skip SharedKeyLite authentication.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\napplication/json\nSat, 30 Oct 2021 12:43:26 GMT\n/testaccount/table1(PartitionKey='Partition1',RowKey='Row1')"
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey testaccount:grDE3K2fkCU2ZF4rSG0qrCGVIOTGofA3vTS1At7AxfY=
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: TableSharedKeyAuthenticator:validate() Signature 1 matched.
2021-10-30T12:43:26.234Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: DeserializerMiddleware: Start deserializing...
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 debug: deserialize(): Raw request body string is (removed all empty characters) {}
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: HandlerMiddleware: DeserializedParameters={"options":{"queryOptions":{},"requestId":"07d64c08-1d1e-4f3b-aac1-46f4f6c814d8","dataServiceVersion":"3.0;","tableEntityProperties":{}},"version":"2017-07-29","body":"ReadableStream"}
2021-10-30T12:43:26.235Z warn: TableHandler:updateEntity() Incoming PartitionKey:Partition1 RowKey:Row1 in URL parameters don't align with entity body PartitionKey:undefined RowKey:undefined.
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 verbose: SerializerMiddleware: Start serializing...
2021-10-30T12:43:26.235Z a40bb2d6-c143-4978-b816-10e4dd967c13 info: EndMiddleware: End response. TotalTimeInMS=2 StatusCode=204 StatusMessage=undefined Headers={"server":"Azurite-Table/3.14.3","x-ms-client-request-id":"07d64c08-1d1e-4f3b-aac1-46f4f6c814d8","x-ms-request-id":"a40bb2d6-c143-4978-b816-10e4dd967c13","x-ms-version":"2020-10-02","date":"Sat, 30 Oct 2021 12:43:26 GMT","etag":"W/\"datetime'2021-10-30T12%3A43%3A26.2330000Z'\""}
```
Log for the failing batch request:
```
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableStorageContextMiddleware: RequestMethod=POST RequestURL=http://testaccount.table.tibp.int/$batch RequestHeaders:{"host":"testaccount.table.tibp.int:10002","accept-charset":"UTF-8","maxdataserviceversion":"3.0;NetFx","accept":"application/json; odata=minimalmetadata","dataserviceversion":"3.0;","x-ms-client-request-id":"a9623db2-17be-46cc-8dc2-dda30b0bdee7","user-agent":"Azure-Cosmos-Table/1.0.8 (.NET CLR 3.1.20; Win32NT 6.2.9200.0)","x-ms-version":"2017-07-29","x-ms-date":"Sat, 30 Oct 2021 12:44:39 GMT","authorization":"SharedKey testaccount:mtibvkoyBfaUZon4Hi4okrA2Axyv+s23qof3fJQLc9U=","content-type":"multipart/mixed; boundary=batch_a68af5c4-e290-4bd9-be51-1687e038e0a4","content-length":"579"} ClientIP=127.0.0.1 Protocol=http HTTPVersion=1.1
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb debug: tableStorageContextMiddleware: Dispatch pattern string: /$batch
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: tableStorageContextMiddleware: Account=testaccount tableName=$batch
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb verbose: DispatchMiddleware: Dispatching request...
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: DispatchMiddleware: Operation=Table_Batch
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyLiteAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyLiteAuthenticator:validate() Request doesn't include valid authentication header. Skip SharedKeyLite authentication.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() [STRING TO SIGN]:"POST\n\nmultipart/mixed; boundary=batch_a68af5c4-e290-4bd9-be51-1687e038e0a4\nSat, 30 Oct 2021 12:44:39 GMT\n/testaccount/$batch"
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey testaccount:mtibvkoyBfaUZon4Hi4okrA2Axyv+s23qof3fJQLc9U=
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: TableSharedKeyAuthenticator:validate() Signature 1 matched.
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb verbose: DeserializerMiddleware: Start deserializing...
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: HandlerMiddleware: DeserializedParameters={"version":"2017-07-29","options":{"requestId":"a9623db2-17be-46cc-8dc2-dda30b0bdee7","dataServiceVersion":"3.0;"},"multipartContentType":"multipart/mixed; boundary=batch_a68af5c4-e290-4bd9-be51-1687e038e0a4","contentLength":579,"body":"ReadableStream"}
2021-10-30T12:44:39.165Z 2ea2e0c5-a40a-4738-9458-943c37467dcb debug: TableHandler:batch() Raw request string is "--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4\r\nContent-Type: multipart/mixed; boundary=changeset_84de5d9b-4978-431f-a447-d6e9405fa332\r\n\r\n--changeset_84de5d9b-4978-431f-a447-d6e9405fa332\r\nContent-Type: application/http\r\nContent-Transfer-Encoding: binary\r\n\r\nPUT http://testaccount.table.tibp.int:10002/table1(PartitionKey='Partition1',RowKey='Row2') HTTP/1.1\r\nAccept: application/json;odata=minimalmetadata\r\nContent-Type: application/json\r\nDataServiceVersion: 3.0;\r\n\r\n{}\r\n--changeset_84de5d9b-4978-431f-a447-d6e9405fa332--\r\n--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4--\r\n"
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb error: ErrorMiddleware: Received an error, fill error information to HTTP response
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb error: ErrorMiddleware: ErrorName=Error ErrorMessage=Couldn't extract path from URL in sub-Request:
PUT http://testaccount.table.tibp.int:10002/table1(PartitionKey='Partition1',RowKey='Row2') HTTP/1.1
Accept: application/json;odata=minimalmetadata
Content-Type: application/json
DataServiceVersion: 3.0;
{}
--changeset_84de5d9b-4978-431f-a447-d6e9405fa332--
--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4--
ErrorStack="Error: Couldn't extract path from URL in sub-Request:\n \r\n\r\nPUT http://testaccount.table.tibp.int:10002/table1(PartitionKey='Partition1',RowKey='Row2') HTTP/1.1\r\nAccept: application/json;odata=minimalmetadata\r\nContent-Type: application/json\r\nDataServiceVersion: 3.0;\r\n\r\n{}\r\n--changeset_84de5d9b-4978-431f-a447-d6e9405fa332--\r\n--batch_a68af5c4-e290-4bd9-be51-1687e038e0a4--\r\n\n at C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\batch\\TableBatchSerialization.js:70:23\n at Array.map ()\n at TableBatchSerialization.deserializeBatchRequest (C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\batch\\TableBatchSerialization.js:57:45)\n at TableBatchOrchestrator.processBatchRequestAndSerializeResponse (C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\batch\\TableBatchOrchestrator.js:44:32)\n at TableHandler.batch (C:\\Users\\administrator\\AppData\\Roaming\\npm\\node_modules\\azurite\\dist\\src\\table\\handlers\\TableHandler.js:466:50)\n at runMicrotasks ()\n at processTicksAndRejections (internal/process/task_queues.js:95:5)"
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb error: ErrorMiddleware: Set HTTP code: 500
2021-10-30T12:44:39.166Z 2ea2e0c5-a40a-4738-9458-943c37467dcb info: EndMiddleware: End response. TotalTimeInMS=1 StatusCode=500 StatusMessage=undefined Headers={"server":"Azurite-Table/3.14.3"}
```
### Have you found a mitigation/solution?
No. The only workaround is to only use single operations.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.