Azure / Azure/AzureCosmosR

query speed issue -- 1.7 million records take over 50 mins

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
R
Stars
6
Forks
12
PR merge metrics
No merged PRs in 30d

Description

on Azure Synapse, we are trying to query cosmosDB (collection contains 1.7 million records) using AzureCosmosR library. To get 1.7 million records (with a partition_key) is taking over 50 min consistently.
 
Steps -
 
devtools::install_github("Azure/AzureCosmosR")
library(AzureCosmosR)
 
endp <- cosmos_endpoint(
    "[https://xxxxxxx.documents.azure.com:443/"](https://xxxxxxx.documents.azure.com/%22),
    key="xxxxxxxxxxxxxxxxxxx"
)
# all databases in this account
list_cosmos_databases(endp)
 

db <- get_cosmos_database(endp, "internal")
# get a specific container - user, event, message, chat etc
container <- get_cosmos_container(db, "message", partition_key="id")
 

df_lst <- query_documents(container,
    "SELECT * FROM container c")
 
 
Are there any optimizations that can be done to fix the slowness of reading the records from cosmosDB account?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.