Use iterators and discourage explicit pagination
- Dominant language
- JavaScript
- Stars
- 7
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Description
Getters accept `limit` and `offset` parameters, which does not encourage the client to implement API pagination properly. For example, in the codebase for ChRIS_ui and chris_ui_ssr we see in many places where `{limit: 100*}` is given as a parameter.
```
$ grep -Fnr 'limit: 100' ChRIS_ui/src chris_ui_ssr/src
ChRIS_ui/src/api/common/index.ts:203: limit: 100,
ChRIS_ui/src/api/common/index.ts:220: limit: 1000,
ChRIS_ui/src/components/catalog/PluginCatalog.tsx:54: const plugins = await plugin.getPlugins({ limit: 1000 });
ChRIS_ui/src/components/feed/AddNode/GuidedConfig.tsx:63: limit: 1000,
ChRIS_ui/src/components/feed/AddPipeline/AddPipeline.tsx:82: limit: 1000,
ChRIS_ui/src/components/feed/AddTsNode/index.tsx:33: limit: 1000,
ChRIS_ui/src/components/feed/CreateFeed/utils/pipelines.ts:8: limit: 100,
ChRIS_ui/src/components/feed/CreateFeed/utils/pipelines.ts:25: limit: 1000,
ChRIS_ui/src/components/feed/NodeDetails/NodeDetails.tsx:61: limit: 100,
ChRIS_ui/src/components/feed/NodeDetails/NodeDetails.tsx:67: limit: 100,
ChRIS_ui/src/components/feed/Pipelines/GeneralCompute.tsx:25: limit: 100,
ChRIS_ui/src/components/feed/Pipelines/ConfigurationPage.tsx:50: limit: 1000,
ChRIS_ui/src/pages/DataLibrary/components/UserLibrary/BrowserContainer.tsx:122: limit: 100,
ChRIS_ui/src/pages/DataLibrary/components/UserLibrary/index.tsx:144: limit: 1000,
ChRIS_ui/src/pages/DataLibrary/components/UserLibrary/index.tsx:268: limit: 1000,
ChRIS_ui/src/pages/SinglePluginPage/SinglePlugin.tsx:46: limit: 1000,
ChRIS_ui/src/store/resources/saga.ts:42: const params = { limit: 100, offset: 0 };
chris_ui_ssr/src/lib/utilities/library/index.ts:44: limit: 10000000,
chris_ui_ssr/src/routes/(_main)/data/[...slug]/+page.server.ts:37: limit: 1000000,
chris_ui_ssr/src/routes/api/downloads/+server.ts:25: limit: 100000
chris_ui_ssr/src/routes/api/metadata/+server.ts:19: const files = await pathList.getFiles({ limit: 100000 });
```
This practice causes performance problems and code quality issues (pagination parameters are often specified in the code where pagination is not relevant in the context)
My suggestion is to provide an iterator (async iterable, stream, whatever it's called) based interface similar to how it is implemented in aiochris and the rust chris client.
https://fnndsc.github.io/aiochris/v0.4.0/aiochris.html#Search
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the listed limit call sites in ChRIS_ui/src and chris_ui_ssr/src, using the provided grep command to review how getters are consumed. Read the aiochris Search documentation and compare it with the rust client approach mentioned in the issue. Done would mean an agreed iterator-based interface that addresses these pagination usages without relying on arbitrary limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, typescript
- Domain
- api, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100