langgenius / langgenius/dify

[Bug]: console dataset list has_more uses raw limit while query caps at 100 (missed siblings of #41784/#41875)

Open
#42,024 1 comment 1 reaction 1 assignee Claimed by @weike-zhang View on GitHub
Dominant language
TypeScript
Stars
156k
Forks
24.6k
Avg merge
22h 9m
Merged PRs (30d)
610

Description

### Expected Behavior

Console dataset list endpoints must report `has_more` based on the page size the server actually used, so clients know whether another page exists.

### Current Behavior

The console **datasets** controllers still compute `has_more = len(items) == limit` while their queries are served through `paginate_query(..., max_per_page=100)` (or, for the by-ids trial endpoint, `max_per_page=len(ids)`). Same bug class as #41780/#41776 (dataset/document/segment Service API lists) and #41875 (annotation lists), which were fixed for those endpoints but missed these console siblings.

Two user-visible behaviours, in five console endpoints:

- **`limit` above the server cap with remaining rows** → `has_more=false`: e.g. `GET /datasets?limit=200` with 150 datasets returns 100 rows but `has_more=false`, so pagination silently stops and 50 rows are never fetched.
- **Last page exactly full** → `has_more=true` (phantom): clients render a "load more" spinner that then returns nothing.

Affected endpoints:
1. `GET /console/api/datasets` — `controllers/console/datasets/datasets.py:577`
2. `GET /console/api/datasets/{dataset_id}/queries` — `datasets.py:876`
3. `GET /console/api/datasets/{dataset_id}/documents` — `datasets_document.py:522`
4. `GET /console/api/datasets/external-knowledge-api` — `external.py:176`
5. `GET /console/api/explore/datasets` (trial, by-ids) — `explore/trial.py:918`

### Steps to Reproduce

Seed 150 datasets in a tenant, then:

```bash
curl -H "Authorization: Bearer " "http://localhost:5001/console/api/datasets?page=1&limit=200"
```

Response (with a 100-row server cap): `{"data": [...100 items...], "has_more": false, "limit": 200, "total": 150, ...}` — pagination stops at page 1 despite 50 remaining rows.

The same applies to documents and external-knowledge-api lists; for the trial endpoint, requesting exactly `limit` ids that all exist returns `has_more=true` with no further page.

### Root Cause

`paginate_query(..., max_per_page=100)` caps the effective page size at 100, but the controllers compare the returned row count against the raw incoming `limit` and echo that raw `limit` back. The fix applied in #41784/#41776 (Service API) and #41875 (annotation) computes an `effective_limit = min(limit, 100)` and `has_more = page * effective_limit < total`; these console endpoints were missed.

### Suggested Fix

Mirror the accepted pattern in the five controllers:

```python
effective_limit = min(limit, 100) # trial: min(limit, len(ids))
items, total = service.get(... per_page=effective_limit ...)
"has_more": page * effective_limit < total
"limit": effective_limit
```

### Environment

- dify main @ d60873d (2026-09-09)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.