Azure / Azure/azure-sdk-for-python

[Cosmos][Query] Support for mixed-type order by queries

Aperta
#43,965 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Client Cosmos cosmos-python-p0 needs-team-attention Service Attention
Lingua principale
Python
Stelle
5.6k
Fork
3.4k
Merge medio
2g
PR unite (30g)
217

Descrizione

From a user:

while doing some tests I came across an odd behavior. I have a collection with millions of docs where all but one have "value" property defined and this property is always a string (see below). The single document where this property is not defined appears to break ORDER BY clause. The specific error I am running into is "Expected NoValue, but got string. Cannot execute cross partition order-by queries on mixed types. Consider filtering your query using IS_STRING or IS_NUMBER to get around this exception."

Why does query engine or SDK consider missing property a "mixed type"? This feels very unintuitive. After some searching, I came across https://github.com/Azure/azure-cosmos-dotnet-v3/issues/733 which seems to refer to the exact same issue. However, the discussion also references two different PRs from some years ago that mention the issue was fixed. Is there a recent regression or has it actually never worked/been fully fixed?

I ran the queries using Portal as well as Python SDK with same result. My queries:

SELECT TOP 20 * FROM c ORDER BY c.value ASC -- this fails with "Expected string, but got NoValue. Cannot execute cross partition order-by queries on mixed types. Consider filtering your query using IS_STRING or IS_NUMBER to get around this exception."
SELECT TOP 20 * FROM c ORDER BY c.value DESC -- this runs fine
SELECT TOP 20 * FROM c WHERE IS_DEFINED(c.value) ORDER BY c.value ASC -- this runs fine
SELECT TOP 20 * FROM c WHERE IS_DEFINED(c.value) ORDER BY c.value DESC -- this runs fine
SELECT DISTINCT c.value FROM c -- for reference here's output of distinct [{"value": "20"}, {"value": "21"}, {}] note that there's empty object because one doc doesn't have value defined at all.

SELECT TOP 20 * FROM c WHERE (c.id = 'some-id') OR (c.id = 'some-id2') ORDER BY c.value ASC -- this runs fine; doc id 'some-id' is the test doc without value and query returns it first in the result set; note both docs belong to the same physical partition
SELECT TOP 20 * FROM c WHERE (c.id = 'some-id') OR (c.id = 'some-id2') ORDER BY c.value DESC -- just for sanity check, this works too; query returns doc 'some-id' last in the result set
SELECT TOP 20 * FROM c WHERE (c.id = 'some-id') OR (c.id = 'some-id3') ORDER BY c.value ASC -- this is same as example before last one, but it fails; the difference is now each doc comes from different partition and somehow SDK or engine can't figure it out;
SELECT TOP 20 * FROM c WHERE (c.id = 'some-id') OR (c.id = 'some-id3') ORDER BY c.value DESC -- this fails too

We currently call this out as a limitation in our SDK README: https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/cosmos/azure-cosmos/README.md#data-plane-limitations. This would be the official tracking item for this work.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Leggi sdk/cosmos/azure-cosmos/README.md, in particolare la sezione sulle limitazioni della Data Plane, quindi riproduci i casi ORDER BY elencati tramite il Python SDK o Portal, confrontando i valori mancanti, il filtraggio con IS_DEFINED e il comportamento tra partizioni. Il lavoro è completato quando i casi ORDER BY tra partizioni con tipi misti non falliscono più e gli esempi nella stessa partizione e gli esempi filtrati continuano a comportarsi come descritto.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
azure, python, sql
Ambito
api, databases
Tipo di issue
Bug
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.