silverstripe / silverstripe/developer-docs

Limit with DataList::filterByCallback is inconsistent.

Open
#14 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

affects/v4 complexity/low impact/low type/docs
Dominant language
No language data
Stars
7
Forks
74
Avg merge
1d 14m
Merged PRs (30d)
6

Description

Affected Version: 4.9+

Description

If you call limit on a DataList, and then call filterByCallback, the data query will first be executed, which gets a limited number of results from the database. This limited result set is then filtered using the callback, which may result in even less results being returned.

This was pointed out by @christopherdarling in the below comment, which perhaps explains it a little more clearly:

that's true but this does unearth the issue of the order you call these methods for example;

  1. if you do Page::get()->filterByCallback(fn () => true)->limit(10)->count();, filterByCallback() loops over the entire set of Page rows (possibly 1000's) to do the filter comparison even after the 10th valid item is found which isn't necessary. I think this code change avoids that
  2. if you do Page::get()->limit(10)->filterByCallback(fn ($item) => $item->URLSegment !== 'home')->count(); I get 9 records because we're filtering on the 10 records from the database and 1 fails this filter
  3. if you do Page::get()->reverse()->limit(10)->filterByCallback(fn ($item) => $item->URLSegment !== 'home')->count(); I get 10 records (where the filter doesn't remove any rows)

Originally posted by @christopherdarling in https://github.com/silverstripe/silverstripe-framework/issues/10248#issuecomment-1060502492

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the DataList limit and filterByCallback entry points, then reproduce the three Page::get() examples from the issue. Compare the returned counts and query order; done means the documented limit and callback combinations produce consistent, expected results without unnecessary filtering.

Written by the indexing model from the issue text.

Assessment

Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.