silverstripe / silverstripe/developer-docs
Limit with DataList::filterByCallback is inconsistent.
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 7
- Forks
- 74
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 6
Description
Affected Version: 4.9+
Description
If you call limit on a DataList, and then call filterByCallback, the data query will first be executed, which gets a limited number of results from the database. This limited result set is then filtered using the callback, which may result in even less results being returned.
This was pointed out by @christopherdarling in the below comment, which perhaps explains it a little more clearly:
that's true but this does unearth the issue of the order you call these methods for example;
- if you do
Page::get()->filterByCallback(fn () => true)->limit(10)->count();,filterByCallback()loops over the entire set of Page rows (possibly 1000's) to do the filter comparison even after the 10th valid item is found which isn't necessary. I think this code change avoids that- if you do
Page::get()->limit(10)->filterByCallback(fn ($item) => $item->URLSegment !== 'home')->count();I get 9 records because we're filtering on the 10 records from the database and 1 fails this filter- if you do
Page::get()->reverse()->limit(10)->filterByCallback(fn ($item) => $item->URLSegment !== 'home')->count();I get 10 records (where the filter doesn't remove any rows)
Originally posted by @christopherdarling in https://github.com/silverstripe/silverstripe-framework/issues/10248#issuecomment-1060502492
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the DataList limit and filterByCallback entry points, then reproduce the three Page::get() examples from the issue. Compare the returned counts and query order; done means the documented limit and callback combinations produce consistent, expected results without unnecessary filtering.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100