[DataSources] More flexible pre-fetching in lazy APIs
- Dominant language
- TypeScript
- Stars
- 247
- Forks
- 78
- Avg merge
- 14h 27m
- Merged PRs (30d)
- 15
Description
## Description
In LazyDataSources, api callback is designed to be simple, and return absolutely minimal set of items required for UI.
However, there are cases when app might need to eagerly load more items than required:
# Case 1. Load whole list/tree in a single request
Currently, API allows fetching more items than requested in case of flat lists. However, it's not possible to hierarchical structures to avoid querying each branch separately.
This can be a good idea to fetch whole tree, if the tree or list is small enough. E.g. some implementations can return top 10 found items if search is active.
This sounds like a case for AsyncDataSource, but it isn't - as we still don't want client-side sorting and filtering.
# Case 2. Preload children for particular node
This can be beneficial if:
- we know UI would need children anyway (e.g., everything is unfolded by default)
- we need to query children when loading parents. E.g. if we implement search w/o flattenSearch option, the server need to scan all children first, and then build a tree from found nodes
# Case 3. Preload all parents for an item.
In case when 'flattenSearch' is enabled, we need to load all parents for all nodes. Server can prefetch them to prevent issuing a separate API calls.
We also load all parents for checked/selected items, to correctly set parent's checkboxes into intermediate state, to indicate that it has children selected. We need to load all parents to do this in cases if parent is folded or children below list's visible area and is not yet loaded.
## Describe the solution you'd like
There are several ideas on how to implement the API for this.
# Option 1: Allow API to return children lists
LazyDataSourceAPI response should be extended with:
{
...
children: [parentId]: { items: TItem[], count: number }
}
Solves:
1. Load whole list/tree in a single request
2. Preload children for particular node
Doesn't solve:
Cons:
- too complex response structure to understand
- we don't expect server responses to match the response structure, so mapping code will be needed
- doesn't solve "Case 3: Load all parents recursively"
# Option 2: Allow each item to have children as a field
Some APIs return children directly in item: { items: [ { id: 1, name: 'parent', children: [ { id: 2, name: 'child' } ] } }
We can support this case with some additional options, like 'getChildren' callback.
Cons:
- I have no good ideas on how to load children only for some nodes, or only in particular cases. E.g. server might want to return children only for top 10 nodes, and rely on lazy-loading for others
- This doesn't handle cases, when children themselves needs to be preloaded on-demand (e.g. only first 20 of 100)
- doesn't solve "Case 3: Load all parents recursively"
This case, can be implemented even w/o any changes: https://codesandbox.io/s/uui-lazy-data-source-prefetch-children-forked-5qss3x?file=/Example.tsx
# Option 3: Allow prefetch any random items set
We can add { prefetch: TItem[] } } to existing LazyDataSourceApiResponse
Loaded items are added to byId list.
This can be used to preload parents (solving Case 3). However, we can't solve other cases with this.
# Option 4. Mix options 1 and 3.
We can add something like this to the LazyDataSource response:
{
prefetchChildren: { [parentId]: LazyDataSourceApiResponse }
prefetchParents: TItem[]
}
This would solve all 3 Cases, but it shares all cons from #1
Let's freeze this for now, until we'll find a good API decision.
Contributor guide
Assessment
This issue has not been assessed yet.