Dynamic model fails with EnableQuery and filtering.

I'm implementing an OData api for a backend with dynamic model. The backend has one or more types with only one common property, an Id. The rest is defined by configuration and I would like to present those configuration changes without having to create new versions of the api. I have managed to create and reflect the EdmModel dynamically, but when I try to use a filter I get an exception using a dictionary for dynamic properties.

I have attached a project that reproduces the problem.

ODataDynamic2.zip

When I create a request to "http://localhost:28038/v1/CustomType1s?$filter=Title eq 'Test1'" I get the following error

System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindPropertyAccessExpression(SingleValueOpenPropertyAccessNode openNode, PropertyInfo prop)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindDynamicPropertyAccessQueryNode(SingleValueOpenPropertyAccessNode openNode)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindSingleValueNode(SingleValueNode node)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.Bind(QueryNode node)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindConvertNode(ConvertNode convertNode)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindSingleValueNode(SingleValueNode node)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.Bind(QueryNode node)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindBinaryOperatorNode(BinaryOperatorNode binaryOperatorNode)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindSingleValueNode(SingleValueNode node)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.Bind(QueryNode node)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindExpression(SingleValueNode expression, RangeVariable rangeVariable, Type elementType)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.BindFilterClause(FilterBinder binder, FilterClause filterClause, Type filterType)
   at Microsoft.AspNetCore.OData.Query.Expressions.FilterBinder.Bind(IQueryable baseQuery, FilterClause filterClause, Type filterType, ODataQueryContext context, ODataQuerySettings querySettings)
   at Microsoft.AspNetCore.OData.Query.FilterQueryOption.ApplyTo(IQueryable query, ODataQuerySettings querySettings)
   at Microsoft.AspNetCore.OData.Query.ODataQueryOptions.ApplyTo(IQueryable query, ODataQuerySettings querySettings)
   at Microsoft.AspNetCore.OData.Query.EnableQueryAttribute.ApplyQuery(IQueryable queryable, ODataQueryOptions queryOptions)
   at Microsoft.AspNetCore.OData.Query.EnableQueryAttribute.ExecuteQuery(Object responseValue, IQueryable singleResultCollection, ControllerActionDescriptor actionDescriptor, HttpRequest request)
   at Microsoft.AspNetCore.OData.Query.EnableQueryAttribute.OnActionExecuted(ActionExecutedContext actionExecutedContext, Object responseValue, IQueryable singleResultCollection, ControllerActionDescriptor actionDescriptor, HttpRequest request)
   at Microsoft.AspNetCore.OData.Query.EnableQueryAttribute.OnActionExecuted(ActionExecutedContext actionExecutedContext)
   at Microsoft.AspNetCore.Mvc.Filters.ActionFilterAttribute.OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|24_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Logged|17_1(ResourceInvoker invoker)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Am I missing something crucial or should it be done in a different way if it's even possible?