OData / OData/AspNetCoreOData

EnsureStableOrdering with GUID column and SkipToken causes strange behavior and errors

Open
#212 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
505
Forks
186
PR merge metrics
No merged PRs in 30d

Description

Versions:

  • Microsoft.AspNetCore.OData - 7.5.8
  • Microsoft.AspNetCore.OData.Versioning.ApiExplorer - 5.0.0
  • AutoMapper - 10.1.1 (probably irrelevant)
  • AutoMapper.Extensions.ExpressionMapping - 4.1.1 (probably irrelevant)
  • EF Core 5.0.7
  • .NET Core 3.1.14

Scenario:

I have an EF Core application that has a table with integer IDs (the primary key) and a GUID field. I want to use the GUID in my EdmModel and hide the integer ID. I want to use EnsureStableOrdering = true and use the GUID as the Key property, and use SkipToken based paging.

When I do all this, when using the nextLink at the bottom of the first page of results, the returned second page does not have a full page of records (1000 records in this case), and all records are kind of sorted as after the $skiptoken=guid-... value ($skiptoken=guid-fd... includes nothing but f... values but some f9..., fa..., and fe..., so neither lexically or numerically consistent). Worse, if I keep following the nextLink, I eventually reach a "page" with zero results, but with a nextLink with a $skip= parameter instead of $skiptoken parameter...and since $skip parameters aren't supported by my service it throws an invalid URL error.

It looks like this may be related to differences in the way MSSQL sorts GUID columns, which is a well-documented oddity. Importantly, using SQL Server Profiler, I captured the SQL query that is sent for the second page, and when I run it directly against the DB, I get a full page worth of results...so it seems that some filtering happens within the OData pipeline that eliminates records. Also importantly, if I change the EdmModel object's Guid column to be of type String (and make AutoMapper translate it accordingly-- not changing the underlying DB or EF entity model) everything seems to work as expected.

Notably, I'd rather be able to specify a different column for EnsureStableOrdering to sort by and probably avoid the issue, but I don't think that's possible.

Startup.cs / Configure:

            modelBuilder.ModelBuilderFactory = () =>
            {
                var aBuilder = new ODataConventionModelBuilder();
                aBuilder.EnableLowerCamelCase(); 

                aBuilder.EntitySet<RedactedSafetyEvent>(nameof(RedactedSafetyEvent))
                    .EntityType.HasKey(m => m.Guid);

                return aBuilder;
            };

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllerRoute(
                    name: "default",
                    pattern: "{controller=Home}/{action=Index}/{id?}");
                endpoints.EnableDependencyInjection();
                endpoints.Select().Expand().Filter().OrderBy().MaxTop(null).Count().SkipToken(); // Default OData endpoints to SkipToken paging
                endpoints.MapVersionedODataRoute("odata", "odata", modelBuilder);
                endpoints.SetTimeZoneInfo(TimeZoneInfo.Utc);
            });
    public class RedactedSafetyEvent
    {
        //public int Id { get; set; }
        public Guid Guid { get; set; }
        //public String Guid { get; set; }
        public DateTime? ReportDate { get; set; }
        public DateTime? IncidentDate { get; set; }
        public bool? IsInjury { get; set; }
        public bool IsFatality { get; set; }
        public DateTime? ClosedDate { get; set; }
    }
    public class RedactedSafetyEventProfile : Profile
    {
        public RedactedSafetyEventProfile()
        {
            CreateMap<SafetyEvent, RedactedSafetyEvent>()
                .ForMember(dst => dst.IsFatality, opt => opt.MapFrom(
                    src => src.InjuriesIllnesses.Any(ii => ii.IsFatality == true)));
                //.ForMember(dst => dst.Guid, opt => opt.MapFrom(src => src.Guid.ToString()));
            CreateMap<RedactedSafetyEvent, SafetyEvent>();
        }
    }

https://localhost:5001/odata/RedactedSafetyEvent?$skiptoken=guid-ffe4ed4c-1d0e-438b-ad86-4629c7dc062a

{"@odata.context":"https://localhost:5001/odata/$metadata#RedactedSafetyEvent","value":[],"@odata.nextLink":"https://localhost:5001/odata/RedactedSafetyEvent?$skip=1000"}
    [ODataRoutePrefix(nameof(RedactedSafetyEvent))]
    public class RedactedSafetyEventController : ODataApiController
    {
        public RedactedSafetyEventController(AppContext context, IMapper mapper) : base(context, mapper)
        {
            // In base class:
            // _context = context;
            // _mapper = mapper;
        }

        [HttpGet]
        [ODataRoute]
        [ProducesResponseType(typeof(ODataValue<IEnumerable<RedactedSafetyEvent>>), StatusCodes.Status200OK)]
        public IEnumerable<RedactedSafetyEvent> Get(ODataQueryOptions<RedactedSafetyEvent> query)
        {
            var data = _context.SafetyEvents.AsQueryable();

            var dtoprojection = data.UseAsDataSource(_mapper).For<RedactedSafetyEvent>();

            var queried = query.ApplyTo(dtoprojection, new ODataQuerySettings { PageSize = 1000, EnsureStableOrdering = true });

            return queried.Cast<RedactedSafetyEvent>();
        }
    }

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the paging behavior in RedactedSafetyEventController, focusing on query.ApplyTo with PageSize 1000, EnsureStableOrdering, and SkipToken. Compare the SQL captured for the second page with the OData pipeline's returned results, then verify that following nextLink produces full pages without an unsupported $skip fallback.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, sql
Domain
api, backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.