[Notion Integration] Support paginated page listing in Knowledge Base import
- Dominant language
- TypeScript
- Stars
- 156k
- Forks
- 24.6k
- Avg merge
- 22h 9m
- Merged PRs (30d)
- 610
Description
### Self Checks
- [x] I have read the [Contributing Guide](https://github.com/langgenius/dify/blob/main/CONTRIBUTING.md) and [Language Policy](https://github.com/langgenius/dify/issues/1542).
- [x] I have searched for existing issues [search for existing issues](https://github.com/langgenius/dify/issues), including closed ones.
- [x] I confirm that I am using English to submit this report, otherwise it will be closed.
- [x] Please do not modify this template :) and fill in all the required fields.
### 1. Is this request related to a challenge you're experiencing? Tell me about your story.
When importing Notion pages into a Knowledge Base, the page selector fetches
all pages at once before returning any results. For large Notion workspaces,
this causes a long wait with no visual feedback — making the experience feel
unresponsive or broken.
**Observed latency:** ~60 s for a workspace with ~4,500 pages.
**Before:**
1. Frontend calls `GET /notion/pre-import/pages` with no pagination parameters
2. Dify API forwards the request to the Notion datasource plugin
3. Plugin calls the Notion `/search` API in a loop until **all** pages are fetched
4. All pages are returned in a single response
5. The page selector displays nothing until the full list is ready
For a workspace with hundreds of pages, step 3 alone can take several seconds,
leaving the user staring at an empty selector with no feedback.
**After:**
1. Frontend calls `GET /notion/pre-import/pages?page_size=50`
2. Dify API forwards `page_size` (and `start_cursor` if present) to the plugin
3. Plugin fetches only one batch from Notion and returns a `next_cursor` if more pages exist
4. Frontend receives the first batch immediately and displays it
5. Frontend automatically requests the next batch until `next_cursor` is absent
The page selector becomes populated within the first network round-trip, and
remaining pages appear progressively in the background.
### 2. Additional context or comments
This improvement spans two repositories and would be delivered as two separate PRs:
**PR 1 — [dify-official-plugins](https://github.com/langgenius/dify-official-plugins):**
- Support optional `page_size` and `start_cursor` parameters in the Notion datasource
- Return a cursor for the next page when more results are available
- Fall back to full fetch when parameters are absent (backward compatible with old Dify core)
**PR 2 — this repository:**
- Add `page_size` / `start_cursor` query parameters to `GET /notion/pre-import/pages`
- Return `next_cursor` in the response
- Migrate the frontend page selector to load pages incrementally
Each PR is independently safe to merge. If PR 1 is not yet deployed, PR 2 falls back
to the existing full-fetch behavior (old plugin ignores unknown parameters and returns all pages;
`next_cursor` is absent, so the frontend behaves exactly as today).
**Acceptance Criteria:**
- [ ] First batch of Notion pages appears without waiting for all pages to load
- [ ] All pages are eventually loaded automatically (no manual "load more" needed)
- [ ] Existing behavior is unchanged when the old plugin version is in use
### 3. Can you help us with this feature?
- [x] I am interested in contributing to this feature.
Contributor guide
Research direction
Start by tracing the frontend calls to `GET /notion/pre-import/pages` and how the page selector consumes the response, then follow the request through the Dify API to the datasource plugin. Done means the selector displays the first batch promptly, automatically loads all remaining pages, and keeps working with the older plugin behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, frontend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100