Optimize api for large datasets

**Feature request**
It would be great to have pagination for core.

**Use case**
Many users need this functionality for production deployments.

**Implementation idea**
Add a new config option `enable_pagination` with default `false`.

Start by locating the core API entry point and configuration handling for `enable_pagination`; the issue does not identify specific files, resources, or response shapes. Define which large-dataset responses are in scope, how pagination should behave, and verify that the default-disabled setting is covered by tests.

Written by the indexing model from the issue text.