angelxmoreno / angelxmoreno/bun-sqlite-orm

[Feature]: Add findPaginated method for efficient data pagination

Open
#58 0 comments 0 reactions 0 assignees View on GitHub
developer-experience feature medium-priority query-builder
Dominant language
TypeScript
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem Statement

Currently, bun-sqlite-orm only provides `find()` which returns all matching records, and `findFirst()` for single records. When working with large datasets, applications need efficient pagination capabilities to:

1. **Avoid memory issues** - Loading thousands of records at once
2. **Improve performance** - Only fetch needed data for current page
3. **Better UX** - Enable paginated interfaces (tables, lists, etc.)
4. **API efficiency** - Reduce payload sizes for web APIs

Without built-in pagination, developers must manually implement `LIMIT`/`OFFSET` logic and calculate pagination metadata.

## Proposed Solution

Add a `findPaginated` static method to `BaseEntity` that returns both the requested page of data and complete pagination metadata.

### Type Definitions

```typescript
interface Pagination {
currentPage: number;
totalPages: number;
totalItems: number;
itemsPerPage: number;
hasNextPage: boolean;
hasPreviousPage: boolean;
startIndex: number;
endIndex: number;
}

interface PaginatedData {
pagination: Pagination;
items: T[];
}

interface PaginationOptions {
page?: number; // default: 1 (1-based pagination)
limit?: number; // default: 25, max: 1000
orderBy?: keyof T; // optional, no default
orderDir?: 'asc' | 'desc'; // default: 'asc' if orderBy specified
conditions?: Where; // Will use proper typing when issue #35 is implemented
}
```

### Method Signature

```typescript
// In BaseEntity
static async findPaginated(
this: new () => T,
options?: PaginationOptions
): Promise>
```

## Example Usage

### Basic Pagination
```typescript
// Default: page 1, limit 25, no ordering
const result = await User.findPaginated();

console.log(result.items); // User[] - first 25 users
console.log(result.pagination.currentPage); // 1
console.log(result.pagination.totalPages); // e.g., 8
console.log(result.pagination.hasNextPage); // true
```

### Advanced Pagination
```typescript
// With all options
const result = await User.findPaginated({
page: 3,
limit: 10,
orderBy: 'createdAt',
orderDir: 'desc',
conditions: { status: 'active' } // Will be Where when #35 is complete
});

// Page 3 shows items 21-30 (1-based pagination, 0-based item indices)
console.log(result.pagination.startIndex); // 21
console.log(result.pagination.endIndex); // 30
console.log(result.items.length); // 10 (or fewer if last page)
```

### Real-world API Usage
```typescript
// API endpoint
app.get('/api/users', async (req, res) => {
const page = parseInt(req.query.page) || 1;
const limit = Math.min(parseInt(req.query.limit) || 25, 100);

const result = await User.findPaginated({
page,
limit,
orderBy: 'name',
orderDir: 'asc',
conditions: req.query.search ? { name: `%${req.query.search}%` } : undefined
});

res.json(result);
});
```

## Implementation Details

### Page Calculation Logic
- **1-based pagination**: Page 1, Page 2, Page 3...
- **0-based item indices**: Page 1 shows items 0-24, Page 2 shows 25-49, etc.
- **SQL OFFSET**: `OFFSET = (page - 1) * limit`

### Limit Validation
```typescript
const safeLimit = Math.min(options.limit || 25, 1000);
```

### SQL Query Structure
```sql
-- Two queries needed:
-- 1. Count total items
SELECT COUNT(*) FROM users WHERE

-- 2. Get paginated data
SELECT * FROM users
WHERE
ORDER BY
LIMIT OFFSET
```

### Pagination Metadata Calculation
```typescript
const totalPages = Math.ceil(totalItems / itemsPerPage);
const startIndex = (currentPage - 1) * itemsPerPage + 1;
const endIndex = Math.min(startIndex + itemsPerPage - 1, totalItems);
const hasNextPage = currentPage < totalPages;
const hasPreviousPage = currentPage > 1;
```

## Alternatives Considered

1. **Cursor-based pagination**: More performant for large datasets but complex to implement and use
2. **Separate `paginate()` method**: Less discoverable than having it on `BaseEntity`
3. **Query builder only**: Would require issue #35 first, but pagination is needed sooner

## Dependencies

- **Issue #35** - Advanced WHERE conditions: Will improve the `conditions` parameter typing from `Record` to proper `Where` type
- **Current query system**: Uses existing `find()` and `count()` infrastructure

## Breaking Changes

None - this is a new method addition.

## Additional Context

### Performance Considerations
- Uses efficient SQL `LIMIT`/`OFFSET` for data retrieval
- Requires additional `COUNT(*)` query for pagination metadata
- Could add option to skip count for performance if total isn't needed

### Type Safety
- Full TypeScript support with generic `` for entity type
- `keyof T` ensures `orderBy` field exists on entity
- Will integrate with issue #35 for typed conditions

### Edge Cases Handled
- Empty result sets (0 items)
- Page numbers beyond available data
- Invalid/negative page numbers
- Limit exceeding maximum allowed

This feature would significantly improve the developer experience for handling large datasets and building paginated interfaces.

Contributor guide

Open the contributing guide

Research direction

Start with BaseEntity and the existing find() and count() infrastructure to understand how queries, conditions, ordering, and entity typing are currently handled. Compare the proposed two-query flow and pagination edge cases with the existing API, while checking issue #35 for the intended WHERE typing. Done means a typed findPaginated method returns items and the specified metadata without breaking existing methods.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, sqlite, typescript
Domain
backend, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.