Add rate limiting for pagination

**Description**
Currently handlers lacks proper retry support.

**Proposed solution**
Add configurable retry with sensible defaults.

**Priority**: Low/Medium

Start by locating the handlers and pagination logic in the repository, then trace how failed requests are currently handled. Define the configurable retry behavior and sensible defaults described in the issue, and verify that pagination failures retry as expected without exceeding the configured limit.

Written by the indexing model from the issue text.