Optimize parsers for large datasets

**Description**
Currently api lacks proper pagination support.

**Proposed solution**
Add configurable pagination with sensible defaults.

**Priority**: Low/Medium

Start by locating the API entry points and parser code involved in handling large datasets, then inspect how current responses are assembled. Clarify which endpoints need pagination, the configurable defaults, and how completion should be verified with large-result cases.

Written by the indexing model from the issue text.