Azure / Azure/azure-sdk-for-java

High RU (1400 times) utilization on native query invoked with CosmosRequestOptions and cosmosAsyncContainer.queryItems than @Query way of invocation

Open
#48,707 6 comments 0 reactions 1 assignee Claimed by @FabianMeiswinkel View on GitHub
Client Cosmos customer-reported needs-team-attention question Service Attention
Dominant language
Java
Stars
2.6k
Forks
2.2k
Avg merge
2d 8h
Merged PRs (30d)
178

Description

I have a cosmos container which is hierarchically partitioned.

`@Data
@NoArgsConstructor
@Container(hierarchicalPartitionKeyPaths = {"/partitionIdLevel1", "/partitionIdLevel2"})
public class UniqueTransactionInbound {

private String id;

private String originalMessageId;

private String messageTimestamp;

private String timeWindow;

private String partitionIdLevel1;

private String partitionIdLevel2;

private Integer ttl;

private Boolean testFlag;

private String hopUuid;

private Integer scenarioVersion;

private String insertTimestamp;

}`

Also, here is a snippet of the indexing policy we have,

`{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/partitionIdLevel1/?"
},
{
"path": "/partitionIdLevel2/?"
},
{
"path": "/testFlag/?"
},
{
"path": "/messageTimestamp/?"
},
{
"path": "/hopUuid/?"
},
{
"path": "/scenarioVersion/?"
},
{
"path": "/timeWindow/?"
}
],
"excludedPaths": [
{
"path": "/*"
},
{
"path": "/\"_etag\"/?"
}
],
"fullTextIndexes": []
}`

I was trying the following queries

`SELECT DISTINCT VALUE c.scenarioVersion FROM c
WHERE c.partitionIdLevel1 = @partitionIdLevel1
AND c.testFlag = @testFlag
AND (c.messageTimestamp BETWEEN @rangeStart AND @rangeEnd)`

`SELECT c.timeWindow, COUNT(1) as count FROM c
WHERE c.partitionIdLevel1 = @partitionIdLevel1
AND c.hopUuid = @hopUuid
GROUP BY c.timeWindow`

In the `@Query` way of executing the query, even though the first level of hierarchical partition was used in the where clause, the CosmosRequestOptions built from the CosmosTemplate did not populate it. We have an issue created in the past for this. (https://github.com/Azure/azure-sdk-for-java/issues/45737)

I also implemented a custom implementation which would populate the first level of the hierarchical partition key on the CosmosRequestOptions.

`PartitionKey partitionKey = new PartitionKeyBuilder().add(partitionIdLevel1).build();
CosmosQueryRequestOptions options = new CosmosQueryRequestOptions();
options.setPartitionKey(partitionKey);
return cosmosAsyncContainer.queryItems(sqlQuerySpec, options, rclass)
.collectList().block();`

The custom implementation was consuming 1400 times the RU that was consumed by the `@Query`. Can someone please help understand the reason for the higher RU consumption and optimize the implementation?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.