Implement pagination to deal with large set of data

https://github.com/Core-Craft/backend/blob/33265bb55ec1c9983d74d0c79d4336df55b9eb8e/app/api/V1/endpoints/user.py#L45

Pagination standards can vary depending on the industry and specific use cases, but there are some common practices and conventions that are often followed in web APIs. Here are some industry-standard conventions for implementing pagination:

Limit and Offset Pagination:
This is one of the most common pagination methods.
Limit: Specifies the maximum number of items to return in a single page.
Offset (or Page Number): Indicates which page of data to retrieve.
Example: /items?limit=10&offset=20 - This request would fetch 10 items starting from the 21st item in the dataset.

Cursor-Based Pagination:
Cursor-based pagination uses a unique identifier (a cursor) to fetch the next page of results.
It's often used when dealing with large datasets and where the order of items can change between requests.
Example: /items?after=cursor_value - This would fetch items after the specified cursor value.

Keyset Pagination:
Keyset pagination is similar to cursor-based pagination but relies on the concept of keys.
Instead of using an offset or page number, it specifies a key, and the API returns items based on the comparison of the key.
Example: /items?start_key=start_value - This would fetch items starting from the item with the specified key.

Page Metadata:
In addition to the actual data, it's common to include metadata in the response to provide information about the pagination, such as the total number of items, the current page, and links to the next and previous pages.

Default Page Sizes:
It's often a good practice to have default page sizes for requests that don't specify a limit explicitly. Common defaults include 10, 20, or 50 items per page.

Sorting Options:
Allow users to specify how the data is sorted, either ascending or descending, and by which attribute.

Consistent Response Format:
Maintain a consistent JSON response format for paginated data to make it easier for clients to parse the data.

Error Handling:
Define clear error responses for cases like invalid page numbers, exceeding limits, or non-existent pages.

Caching and Etag Headers:
Implement caching strategies to improve performance. Use Etag headers to indicate when data has changed.

HATEOAS (Hypermedia as the Engine of Application State):
Consider using HATEOAS to provide links to related pages in the response, allowing clients to navigate the API more easily.