OData / OData/AspNetCoreOData

Error(s) running OData against Cosmos

Open
#738 15 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug investigating
Dominant language
C#
Stars
505
Forks
186
PR merge metrics
No merged PRs in 30d

Description

Describe the bug
I am trying to run OData against a Cosmos database container and while some basic functionality works, when I add $select and other OData actions I start to get a lot of odd errors. I was able to find work arounds for some of them, but these last 2 I seem to be stuck on even when stepping through the Microsoft.Azure.Cosmos source.

Note: I have already posted to the Microsoft.Azure.Cosmos site and they see the issue being that the OData query generator is creating a "Conditional" clause ({IIF(($it == null), null, $it.Changes)}) that cannot be translated to Cosmos SQL: https://github.com/Azure/azure-cosmos-dotnet-v3/issues/3542 . I tried playing with the options HandleNullPropagation and HandleReferenceNavigationPropertyExpandFilter but they did not seem to fix this.

To Reproduce
I have created a repo with code that reproduces all of these issues I have been seeing:

https://github.com/michael-d-atl404/ODataWithCosmosErrors

This project was run with a local Cosmos Emulator running: https://learn.microsoft.com/en-us/azure/cosmos-db/local-emulator. I included setup for that in the README.md.

I also included info in the README.md on the first 3 errors I was able to work around as well as the URLs I am still unable to get to work along with the errors I am getting.

These URLs work just fine:

https://localhost:7253/odata/Audits
https://localhost:7253/odata/Audits?$select=id,user,date,changes

but then these other URLs are causing me errors:

https://localhost:7253/odata/Audits?$count=true&$select=id,user,date,changes&$orderby=date%20desc&$top=50
https://localhost:7253/odata/Audits?$count=true&$select=id,user,date&$expand=changes($select=prop,old,new)&$orderby=date%20desc&$top=50

The real API being used here has OData running on some EFCore resources and some OData. Everything seemed to be running fine on both from our Vue app until I hit this new Cosmos model for Audits where the document had a child array of simple objects. I should be able to make it work with either pulling the entire child collection or navigating it as a "$expand", I just need one of them to work and would prefer that it did work with paging as it is possible there will be a lot of these records.

Basically I debugged down to this exception being thrown from ExpressionToSql.Translate:

Microsoft.Azure.Cosmos.Linq.DocumentQueryException: Expression with NodeType 'Conditional' is not supported.

where it appears to be running against an IIF null check on the "changes" IEnumerable attribute and there is no code to handle that particular expression tree. Not sure if there is another setting to try to bypass this or not. I attempted to use OData Query Option settings:

HandleNullPropagation = Microsoft.AspNetCore.OData.Query.HandleNullPropagationOption.False

and

HandleReferenceNavigationPropertyExpandFilter = true

but neither of these seemed to fix anything.

Expected behavior
Proper, paged OData responses for at least one of those 2 URL patterns.

Actual behavior

I think that the main one I would like fixed is this (for URL "https://localhost:7253/odata/Audits?$count=true&$select=id,user,date,changes&$orderby=date%20desc&$top=50" ):

Full Error Stack:
      Microsoft.Azure.Cosmos.Linq.DocumentQueryException: Expression with NodeType 'Conditional' is not supported., Windows/10.0.19044 cosmos-netstandard-sdk/3.29.4
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.Translate(Expression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMethodCall(MethodCallExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.Translate(Expression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitCollectionExpression(Expression expression, TranslationContext context, String parameterName)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitCollectionExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.CreateSubquery(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitConditional(ConditionalExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberAssignment(MemberAssignment inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitBindingList(ReadOnlyCollection`1 inputExpressionList, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberInit(MemberInitExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberAssignment(MemberAssignment inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitBindingList(ReadOnlyCollection`1 inputExpressionList, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberInit(MemberInitExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberAssignment(MemberAssignment inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitBindingList(ReadOnlyCollection`1 inputExpressionList, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMemberInit(MemberInitExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitNonSubqueryScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitScalarExpression(Expression expression, ReadOnlyCollection`1 parameters, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitSelect(ReadOnlyCollection`1 arguments, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMethodCall(MethodCallExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.Translate(Expression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.VisitMethodCall(MethodCallExpression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.Translate(Expression inputExpression, TranslationContext context)
         at Microsoft.Azure.Cosmos.Linq.ExpressionToSql.TranslateQuery(Expression inputExpression, IDictionary`2 parameters, CosmosLinqSerializerOptions linqSerializerOptions)
         at Microsoft.Azure.Cosmos.Linq.SqlTranslator.TranslateQuery(Expression inputExpression, CosmosLinqSerializerOptions linqSerializerOptions, IDictionary`2 parameters)
         at Microsoft.Azure.Cosmos.Linq.CosmosLinqQuery`1.CreateFeedIterator(Boolean isContinuationExpected)
         at Microsoft.Azure.Cosmos.Linq.CosmosLinqQuery`1.GetEnumerator()+MoveNext()
         at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
         at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
         at Microsoft.Azure.Cosmos.Linq.CosmosLinqQueryProvider.Execute[TResult](Expression expression)
         at System.Linq.Queryable.Count[TSource](IQueryable`1 source)
         at ODataCosmosTest.EnableCosmosQueryAttribute.<>c__DisplayClass0_0.<ApplyQuery>b__0() in C:\Users\michaeld\localProjects\ODataCosmosTest\ODataCosmosTest\EnableCosmosQueryAttribute.cs:line 26
         at Microsoft.AspNetCore.OData.Abstracts.ODataFeature.get_TotalCount()
         at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.CreateResourceSet(IEnumerable resourceSetInstance, IEdmCollectionTypeReference resourceSetType, ODataSerializerContext writeContext)
         at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteResourceSetAsync(IEnumerable enumerable, IEdmTypeReference resourceSetType, ODataWriter writer, ODataSerializerContext writeContext)
         at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteObjectInlineAsync(Object graph, IEdmTypeReference expectedType, ODataWriter writer, ODataSerializerContext writeContext)
         at Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteObjectAsync(Object graph, Type type, ODataMessageWriter messageWriter, ODataSerializerContext writeContext)
         at Microsoft.AspNetCore.OData.Formatter.ODataOutputFormatterHelper.WriteToStreamAsync(Type type, Object value, IEdmModel model, ODataVersion version, Uri baseAddress, MediaTypeHeaderValue contentType, HttpRequest request, IHeaderDictionary requestHeaders, IODataSerializerProvider serializerProvider)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResultFilterAsync>g__Awaited|30_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
      --- End of stack trace from previous location ---
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
         at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
         at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
         at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
         at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
         at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
         at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
         at Microsoft.WebTools.BrowserLink.Net.BrowserLinkMiddleware.ExecuteWithFilterAsync(IHttpSocketAdapter injectScriptSocket, String requestId, HttpContext httpContext)
         at Microsoft.AspNetCore.Watch.BrowserRefresh.BrowserRefreshMiddleware.InvokeAsync(HttpContext context)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Environment summary

I am running on Windows 10 Pro and using the latest .NET 6, Microsoft.AspNetCore.OData 8.0.11 and Microsoft.Azure.Cosmos 3.31.1

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked reproduction project's README.md and run it against the local Cosmos Emulator. Trace the failing OData URLs through EnableCosmosQueryAttribute.cs at line 26 to ExpressionToSql.Translate, using the reported Conditional expression as the entry point. Done means at least one requested URL returns a correctly paged OData response.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, csharp
Domain
api, backend, databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.