> ## Documentation Index
> Fetch the complete documentation index at: https://docs.deploystack.io/llms.txt
> Use this file to discover all available pages before exploring further.

# API Pagination Guide

> Complete guide to implementing pagination in DeployStack Backend APIs, including best practices, patterns, and examples.

This document provides comprehensive guidance on implementing pagination in DeployStack Backend APIs. Pagination is essential for handling large datasets efficiently and providing a good user experience.

## Overview

DeployStack uses **offset-based pagination** with standardized query parameters and response formats. This approach provides:

* **Consistent API Interface**: All paginated endpoints use the same parameter names and response structure
* **Performance**: Reduces memory usage and response times for large datasets
* **User Experience**: Enables smooth navigation through large result sets
* **Scalability**: Handles growing datasets without performance degradation

## Standard Pagination Parameters

### Query Parameters Schema

All paginated endpoints should accept these standardized query parameters:

```typescript theme={null}
const PAGINATION_QUERY_SCHEMA = {
  type: 'object',
  properties: {
    limit: {
      type: 'string',
      pattern: '^\\d+$',
      description: 'Maximum number of items to return (1-100, default: 20)'
    },
    offset: {
      type: 'string',
      pattern: '^\\d+$',
      description: 'Number of items to skip from the beginning (≥0, default: 0)'
    }
  },
  additionalProperties: false
} as const;
```

#### Parameter Details

* **`limit`** (optional, default: 20)
  * Type: String (converted to Number in handler)
  * Range: 1-100
  * Description: Maximum number of items to return
  * Validation: Must be a positive integer between 1 and 100

* **`offset`** (optional, default: 0)
  * Type: String (converted to Number in handler)
  * Range: ≥ 0
  * Description: Number of items to skip from the beginning
  * Validation: Must be a non-negative integer

### Parameter Validation in Handlers

Query parameters are always strings in HTTP. Convert and validate them in your route handlers:

```typescript theme={null}
// Parse and validate pagination parameters
function validatePaginationParams(query: any): { limit: number; offset: number } {
  const limit = query.limit ? parseInt(query.limit, 10) : 20;
  const offset = query.offset ? parseInt(query.offset, 10) : 0;
  
  // Validate limit
  if (isNaN(limit) || limit < 1 || limit > 100) {
    throw new Error('Limit must be between 1 and 100');
  }
  
  // Validate offset
  if (isNaN(offset) || offset < 0) {
    throw new Error('Offset must be non-negative');
  }
  
  return { limit, offset };
}
```

## Standard Response Format

### Response Schema

All paginated endpoints should return responses in this format:

```typescript theme={null}
const PAGINATED_RESPONSE_SCHEMA = {
  type: 'object',
  properties: {
    success: { type: 'boolean' },
    data: {
      type: 'object',
      properties: {
        // Your actual data array (name varies by endpoint)
        items: { 
          type: 'array',
          items: { /* your item schema */ }
        },
        
        // Pagination metadata
        pagination: {
          type: 'object',
          properties: {
            total: { 
              type: 'number',
              description: 'Total number of items available'
            },
            limit: { 
              type: 'number',
              description: 'Items per page (as requested)'
            },
            offset: { 
              type: 'number',
              description: 'Current offset (as requested)'
            },
            has_more: { 
              type: 'boolean',
              description: 'Whether more items are available'
            }
          },
          required: ['total', 'limit', 'offset', 'has_more']
        }
      },
      required: ['items', 'pagination']
    }
  },
  required: ['success', 'data']
} as const;
```

### Response Example

```json theme={null}
{
  "success": true,
  "data": {
    "servers": [
      {
        "id": "server-1",
        "name": "Example Server",
        // ... other server fields
      }
      // ... more servers
    ],
    "pagination": {
      "total": 150,
      "limit": 20,
      "offset": 40,
      "has_more": true
    }
  }
}
```

### Pagination Metadata Fields

* **`total`**: Total number of items available (across all pages)
* **`limit`**: Number of items per page (echoes the request parameter)
* **`offset`**: Current starting position (echoes the request parameter)
* **`has_more`**: Boolean indicating if more items are available after this page

## Implementation Pattern

### 1. Route Schema Definition

```typescript theme={null}
import { type FastifyInstance } from 'fastify';

// Query parameters schema (including pagination)
const QUERY_SCHEMA = {
  type: 'object',
  properties: {
    // Your filtering parameters
    category: { 
      type: 'string',
      description: 'Filter by category'
    },
    status: { 
      type: 'string',
      enum: ['active', 'inactive'],
      description: 'Filter by status'
    },
    
    // Standard pagination parameters
    limit: {
      type: 'string',
      pattern: '^\\d+$',
      description: 'Maximum number of items to return (1-100, default: 20)'
    },
    offset: {
      type: 'string',
      pattern: '^\\d+$',
      description: 'Number of items to skip (≥0, default: 0)'
    }
  },
  additionalProperties: false
} as const;

// Response schema
const RESPONSE_SCHEMA = {
  type: 'object',
  properties: {
    success: { type: 'boolean' },
    data: {
      type: 'object',
      properties: {
        items: {
          type: 'array',
          items: {
            // Your item schema here
            type: 'object',
            properties: {
              id: { type: 'string' },
              name: { type: 'string' },
              // ... other properties
            }
          }
        },
        pagination: {
          type: 'object',
          properties: {
            total: { type: 'number' },
            limit: { type: 'number' },
            offset: { type: 'number' },
            has_more: { type: 'boolean' }
          },
          required: ['total', 'limit', 'offset', 'has_more']
        }
      },
      required: ['items', 'pagination']
    }
  },
  required: ['success', 'data']
} as const;

// TypeScript interfaces
interface QueryParams {
  category?: string;
  status?: 'active' | 'inactive';
  limit?: string;
  offset?: string;
}

interface PaginatedResponse {
  success: boolean;
  data: {
    items: Item[];
    pagination: {
      total: number;
      limit: number;
      offset: number;
      has_more: boolean;
    };
  };
}
```

### 2. Route Handler Implementation

```typescript theme={null}
export default async function listItems(server: FastifyInstance) {
  server.get('/api/items', {
    schema: {
      tags: ['Items'],
      summary: 'List items with pagination',
      description: 'Retrieve items with pagination support. Supports filtering and sorting.',
      
      querystring: QUERY_SCHEMA,
      
      response: {
        200: RESPONSE_SCHEMA,
        400: {
          type: 'object',
          properties: {
            success: { type: 'boolean', default: false },
            error: { type: 'string' }
          }
        }
      }
    }
  }, async (request, reply) => {
    try {
      // Parse and validate query parameters
      const query = request.query as QueryParams;
      const { limit, offset } = validatePaginationParams(query);
      
      // Extract filter parameters
      const filters = {
        category: query.category,
        status: query.status
      };
      
      // Get all items (with filtering applied)
      const allItems = await yourService.getItems(filters);
      
      // Apply pagination
      const total = allItems.length;
      const paginatedItems = allItems.slice(offset, offset + limit);
      
      // Log pagination info
      server.log.info({
        operation: 'list_items',
        totalResults: total,
        returnedResults: paginatedItems.length,
        pagination: { limit, offset }
      }, 'Items list completed');
      
      // Return paginated response
      const response: PaginatedResponse = {
        success: true,
        data: {
          items: paginatedItems,
          pagination: {
            total,
            limit,
            offset,
            has_more: offset + limit < total
          }
        }
      };
      
      const jsonString = JSON.stringify(response);
      return reply.status(200).type('application/json').send(jsonString);
    } catch (error) {
      server.log.error({ error }, 'Failed to list items');
      
      const errorResponse = {
        success: false,
        error: error instanceof Error ? error.message : 'Failed to retrieve items'
      };
      const jsonString = JSON.stringify(errorResponse);
      return reply.status(400).type('application/json').send(jsonString);
    }
  });
}
```

## Database-Level Pagination (Advanced)

For better performance with large datasets, implement pagination at the database level:

### Using Drizzle ORM

```typescript theme={null}
import { desc, asc, sql, eq } from 'drizzle-orm';

async getItemsPaginated(
  filters: ItemFilters,
  limit: number,
  offset: number
): Promise<{ items: Item[], total: number }> {
  // Build base query with filters
  let query = this.db.select().from(items);
  
  // Apply filters
  if (filters.category) {
    query = query.where(eq(items.category, filters.category));
  }
  
  // Get total count (before pagination)
  const countQuery = this.db.select({ count: sql<number>`count(*)` }).from(items);
  // Apply same filters to count query
  if (filters.category) {
    countQuery = countQuery.where(eq(items.category, filters.category));
  }
  const [{ count: total }] = await countQuery;
  
  // Apply pagination and ordering
  const paginatedItems = await query
    .orderBy(desc(items.created_at))
    .limit(limit)
    .offset(offset);
  
  return {
    items: paginatedItems,
    total
  };
}
```

### Updated Route Handler

```typescript theme={null}
// In your route handler
const { items, total } = await yourService.getItemsPaginated(filters, limit, offset);

const response: PaginatedResponse = {
  success: true,
  data: {
    items,
    pagination: {
      total,
      limit,
      offset,
      has_more: offset + limit < total
    }
  }
};

const jsonString = JSON.stringify(response);
return reply.status(200).type('application/json').send(jsonString);
```

## Client-Side Usage Examples

### JavaScript/TypeScript

```typescript theme={null}
interface PaginationParams {
  limit?: number;
  offset?: number;
}

interface PaginatedResponse<T> {
  success: boolean;
  data: {
    items: T[];
    pagination: {
      total: number;
      limit: number;
      offset: number;
      has_more: boolean;
    };
  };
}

async function fetchItems(params: PaginationParams = {}): Promise<PaginatedResponse<Item>> {
  const url = new URL('/api/items', baseUrl);
  
  if (params.limit) url.searchParams.set('limit', params.limit.toString());
  if (params.offset) url.searchParams.set('offset', params.offset.toString());
  
  const response = await fetch(url.toString(), {
    credentials: 'include',
    headers: { 'Accept': 'application/json' }
  });
  
  return await response.json();
}

// Usage examples
const firstPage = await fetchItems({ limit: 20, offset: 0 });
const secondPage = await fetchItems({ limit: 20, offset: 20 });
const customPage = await fetchItems({ limit: 50, offset: 100 });
```

### Vue.js Composable

```typescript theme={null}
import { ref, computed } from 'vue';

export function usePagination<T>(
  fetchFunction: (limit: number, offset: number) => Promise<PaginatedResponse<T>>,
  initialLimit = 20
) {
  const items = ref<T[]>([]);
  const currentPage = ref(1);
  const limit = ref(initialLimit);
  const total = ref(0);
  const loading = ref(false);
  
  const totalPages = computed(() => Math.ceil(total.value / limit.value));
  const hasNextPage = computed(() => currentPage.value < totalPages.value);
  const hasPrevPage = computed(() => currentPage.value > 1);
  
  const offset = computed(() => (currentPage.value - 1) * limit.value);
  
  async function loadPage(page: number) {
    if (page < 1 || page > totalPages.value) return;
    
    loading.value = true;
    try {
      const response = await fetchFunction(limit.value, (page - 1) * limit.value);
      items.value = response.data.items;
      total.value = response.data.pagination.total;
      currentPage.value = page;
    } finally {
      loading.value = false;
    }
  }
  
  async function nextPage() {
    if (hasNextPage.value) {
      await loadPage(currentPage.value + 1);
    }
  }
  
  async function prevPage() {
    if (hasPrevPage.value) {
      await loadPage(currentPage.value - 1);
    }
  }
  
  return {
    items,
    currentPage,
    limit,
    total,
    totalPages,
    loading,
    hasNextPage,
    hasPrevPage,
    loadPage,
    nextPage,
    prevPage
  };
}
```

## Implementation Guidelines

### 1. Consistent Parameter Validation

Always use the same validation rules across all endpoints:

```typescript theme={null}
// Create a reusable validation function
export function validatePaginationParams(query: any): { limit: number; offset: number } {
  const limit = query.limit ? parseInt(query.limit, 10) : 20;
  const offset = query.offset ? parseInt(query.offset, 10) : 0;
  
  if (isNaN(limit) || limit < 1 || limit > 100) {
    throw new Error('Limit must be between 1 and 100');
  }
  
  if (isNaN(offset) || offset < 0) {
    throw new Error('Offset must be non-negative');
  }
  
  return { limit, offset };
}

// Reusable schema constant
export const PAGINATION_QUERY_SCHEMA = {
  type: 'object',
  properties: {
    limit: {
      type: 'string',
      pattern: '^\\d+$',
      description: 'Maximum number of items to return (1-100, default: 20)'
    },
    offset: {
      type: 'string',
      pattern: '^\\d+$',
      description: 'Number of items to skip (≥0, default: 0)'
    }
  },
  additionalProperties: false
} as const;

// Use in your endpoint schemas
const QUERY_SCHEMA = {
  type: 'object',
  properties: {
    // Your specific filters
    category: { type: 'string' },
    status: { type: 'string', enum: ['active', 'inactive'] },
    
    // Include pagination
    ...PAGINATION_QUERY_SCHEMA.properties
  },
  additionalProperties: false
} as const;
```

### 2. Proper Error Handling

```typescript theme={null}
try {
  const { limit, offset } = validatePaginationParams(request.query);
  // ... rest of handler
} catch (error) {
  const errorResponse = {
    success: false,
    error: error instanceof Error ? error.message : 'Invalid query parameters'
  };
  const jsonString = JSON.stringify(errorResponse);
  return reply.status(400).type('application/json').send(jsonString);
}
```

### 3. Performance Considerations

* **Database Pagination**: Use `LIMIT` and `OFFSET` at the database level for large datasets
* **Indexing**: Ensure proper database indexes on columns used for sorting
* **Caching**: Consider caching total counts for frequently accessed endpoints
* **Reasonable Limits**: Enforce maximum page sizes (e.g., 100 items)

### 4. OpenAPI Documentation

Include clear pagination documentation in your API specs:

```typescript theme={null}
schema: {
  tags: ['Items'],
  summary: 'List items with pagination',
  description: `
    Retrieve items with pagination support. 
    
    **Pagination Parameters:**
    - \`limit\`: Items per page (1-100, default: 20)
    - \`offset\`: Items to skip (≥0, default: 0)
    
    **Response includes:**
    - \`data.items\`: Array of items for current page
    - \`data.pagination.total\`: Total items available
    - \`data.pagination.has_more\`: Whether more pages exist
  `,
  // ... rest of schema
}
```

## Common Pitfalls and Solutions

### 1. Inconsistent Response Formats

❌ **Wrong**: Different endpoints use different response structures

```typescript theme={null}
// Endpoint A
{ data: items, total: 100, page: 1 }

// Endpoint B  
{ results: items, count: 100, offset: 20 }
```

✅ **Correct**: Use standardized response format

```typescript theme={null}
// All endpoints
{
  success: true,
  data: {
    items: [...],
    pagination: { total, limit, offset, has_more }
  }
}
```

### 2. Missing Validation

❌ **Wrong**: No parameter validation

```typescript theme={null}
const limit = parseInt(request.query.limit) || 20;
const offset = parseInt(request.query.offset) || 0;
```

✅ **Correct**: Proper validation function

```typescript theme={null}
const { limit, offset } = validatePaginationParams(request.query);
```

### 3. Performance Issues

❌ **Wrong**: Loading all data then slicing

```typescript theme={null}
const allItems = await db.select().from(items); // Loads everything!
const paginated = allItems.slice(offset, offset + limit);
```

✅ **Correct**: Database-level pagination

```typescript theme={null}
const items = await db.select().from(items)
  .limit(limit)
  .offset(offset);
```

### 4. Incorrect Total Count

❌ **Wrong**: Using paginated results length

```typescript theme={null}
const items = await getItemsPaginated(limit, offset);
const total = items.length; // Wrong! This is just current page
```

✅ **Correct**: Separate count query

```typescript theme={null}
const [items, total] = await Promise.all([
  getItemsPaginated(limit, offset),
  getItemsCount(filters)
]);
```

## Real-World Examples

### Example 1: MCP Servers List (Current Implementation)

```typescript theme={null}
// File: services/backend/src/routes/mcp/servers/list.ts
export default async function listServers(server: FastifyInstance) {
  server.get('/mcp/servers', {
    schema: {
      tags: ['MCP Servers'],
      summary: 'List MCP servers',
      description: 'Retrieve MCP servers with pagination support...',
      querystring: QUERY_SCHEMA,
      response: {
        200: LIST_SERVERS_RESPONSE_SCHEMA
      }
    }
  }, async (request, reply) => {
    const { limit, offset } = validatePaginationParams(request.query);
    const filters = extractFilters(request.query);
    
    const allServers = await catalogService.getServersForUser(
      userId, userRole, teamIds, filters
    );
    
    const total = allServers.length;
    const paginatedServers = allServers.slice(offset, offset + limit);
    
    const response = {
      success: true,
      data: {
        servers: paginatedServers,
        pagination: {
          total,
          limit,
          offset,
          has_more: offset + limit < total
        }
      }
    };
    
    const jsonString = JSON.stringify(response);
    return reply.status(200).type('application/json').send(jsonString);
  });
}
```

### Example 2: Search Endpoint (Reference Implementation)

The search endpoint (`/mcp/servers/search`) demonstrates the complete pagination pattern and can serve as a reference for implementing pagination in other endpoints.

## Testing Pagination

### Unit Tests

```typescript theme={null}
describe('Pagination', () => {
  test('should return first page with default limit', async () => {
    const response = await request(app)
      .get('/api/items')
      .expect(200);
    
    expect(response.body.data.pagination).toEqual({
      total: expect.any(Number),
      limit: 20,
      offset: 0,
      has_more: expect.any(Boolean)
    });
  });
  
  test('should handle custom pagination parameters', async () => {
    const response = await request(app)
      .get('/api/items?limit=10&offset=20')
      .expect(200);
    
    expect(response.body.data.pagination.limit).toBe(10);
    expect(response.body.data.pagination.offset).toBe(20);
  });
  
  test('should validate pagination parameters', async () => {
    await request(app)
      .get('/api/items?limit=invalid')
      .expect(400);
      
    await request(app)
      .get('/api/items?limit=101') // Over maximum
      .expect(400);
  });
});
```

### Integration Tests

```typescript theme={null}
test('should paginate through all results', async () => {
  const limit = 5;
  let offset = 0;
  let allItems = [];
  let hasMore = true;
  
  while (hasMore) {
    const response = await request(app)
      .get(`/api/items?limit=${limit}&offset=${offset}`)
      .expect(200);
    
    const { items, pagination } = response.body.data;
    allItems.push(...items);
    
    hasMore = pagination.has_more;
    offset += limit;
  }
  
  // Verify we got all items
  expect(allItems.length).toBe(totalExpectedItems);
});
```