docs(mdbook-htmx): server side search

# Server-Side Search Implementation Guide

This document covers search implementation options for mdbook-htmx deployments, from simple in-memory solutions to enterprise-scale search engines.

---

## Overview

mdbook-htmx generates search indexes at build time. When scopes are enabled, it generates **separate indexes per scope** to prevent information leakage.

### Build Output

```
book/htmx/
├── search-index.json # Full index (all pages)
├── search-index.developers.json # Only developer-scoped pages
├── search-index.managers.json # Only manager-scoped pages
└── search-index.sre.json # Only SRE-scoped pages
```

### Index Schema

```json
{
"config": {
"heading_split_level": 3,
"include_body": true
},
"scope": "developers", // null for full index
"documents": [
{
"path": "/docs/chapter-1",
"title": "Getting Started",
"body": "Full text content...",
"headings": [
{ "level": 2, "text": "Installation", "anchor": "#installation" }
],
"auth": {
"access": "roles",
"roles": ["developer"]
},
"scopes": ["developers", "sre"]
}
]
}
```

The server uses the appropriate scope index and applies authorization filtering.

---

## Security: Scope-Aware & AuthZ-Aware Search

### Why Separate Indexes?

A single index with runtime filtering still **leaks information**:

- Page titles visible to unauthorized users
- Content snippets may reveal sensitive data
- Existence of pages is metadata leakage

**Solution**: Generate scope-specific indexes at build time. Each index only contains documents visible in that scope.

### Defense in Depth: Runtime AuthZ Filtering

Even with scope-specific indexes, apply runtime authorization as defense in depth:

```typescript
app.get("/docs/search", async (c) => {
const query = c.req.query("q");
const scope = c.req.query("scope") || "all";
const user = c.get("user");

// 1. Select scope-specific index
const index = searchIndexes[scope] || searchIndexes.all;

// 2. Search within scope
let results = index.search(query);

// 3. Runtime authorization filter (defense in depth)
results = results.filter((result) => {
const doc = result.item || result;

// Public pages - always visible
if (doc.auth?.access === "public") return true;

// Authenticated pages - require login
if (doc.auth?.access === "authenticated") return !!user;

// Role-based pages - check roles
if (doc.auth?.access === "roles") {
return doc.auth.roles?.some((role) => user?.roles?.includes(role));
}

return false;
});

return c.html(renderSearchResults(results));
});
```

### Anonymous User Search

Anonymous users:

- Can only search the `all` scope (or `public` scope if configured)
- Only see results from `access: public` pages
- Cannot switch to other scopes

```typescript
// Enforce public-only for anonymous users
if (!user) {
scope = "all"; // Force to public scope
}
```

---

## Solution Comparison

| Solution | Type | Latency | Index Size Limit | Self-Hosted | Free Tier | Best For |
| ----------------- | ----------- | ------- | ---------------- | ----------- | --------------- | -------------------------- |
| **Fuse.js** | In-memory | <5ms | ~50MB | Yes | Open source | Small-medium books |
| **MiniSearch** | In-memory | <2ms | ~50MB | Yes | Open source | Small-medium, faster |
| **FlexSearch** | In-memory | <1ms | ~100MB | Yes | Open source | Speed-critical |
| **Pagefind** | Static+WASM | ~20ms | Unlimited | Yes | Open source | Large static sites |
| **Meilisearch** | External | ~10ms | Unlimited | Yes | Cloud free tier | Medium-large, self-hosted |
| **Typesense** | External | ~5ms | Unlimited | Yes | Cloud free tier | Performance-critical |
| **Algolia** | SaaS | ~10ms | Unlimited | No | 10k searches/mo | Enterprise, managed |
| **Elasticsearch** | External | ~50ms | Unlimited | Yes | Open source | Complex queries, analytics |

---

## 1. In-Memory Solutions (Cloudflare Workers Compatible)

These solutions load the entire index into memory. Best for books with <1000 pages.

### 1.1 Fuse.js (Recommended for Simplicity)

**Pros**: Well-documented, fuzzy matching, configurable scoring
**Cons**: Slower on large datasets, ~50KB bundle

```typescript
// worker.ts
import Fuse from "fuse.js";
import searchIndex from "./search-index.json";

const fuse = new Fuse(searchIndex.documents, {
keys: [
{ name: "title", weight: 2.0 },
{ name: "headings.text", weight: 1.5 },
{ name: "body", weight: 1.0 },
],
includeMatches: true,
threshold: 0.3,
ignoreLocation: true,
minMatchCharLength: 2,
});

app.get("/docs/search", async (c) => {
const query = c.req.query("q");
if (!query || query.length < 2) {
return c.html('

Type at least 2 characters

');
}

const results = fuse.search(query, { limit: 10 });

return c.html(`

${results
.map(
(r) => `

• 
  
  ${highlightMatches(r.item.title, r.matches)}
  
  

  ${getExcerpt(r.item.body, r.matches)}

  
  

`,
)
.join("")}

`);
});

function highlightMatches(
text: string,
matches: Fuse.FuseResultMatch[],
): string {
// Highlight matching text with tags
const titleMatch = matches?.find((m) => m.key === "title");
if (!titleMatch) return text;

let result = text;
titleMatch.indices.reverse().forEach(([start, end]) => {
result =
result.slice(0, start) +
"" +
result.slice(start, end + 1) +
"" +
result.slice(end + 1);
});
return result;
}
```

**Deployment Requirements**:

- Bundle `fuse.js` with worker (~50KB)
- Include `search-index.json` in worker assets
- Memory: ~2x index size

### 1.2 MiniSearch (Recommended for Speed)

**Pros**: Faster than Fuse.js, smaller bundle, prefix matching
**Cons**: Less fuzzy matching flexibility

```typescript
import MiniSearch from "minisearch";
import searchIndex from "./search-index.json";

const miniSearch = new MiniSearch({
fields: ["title", "body"],
storeFields: ["title", "path"],
searchOptions: {
boost: { title: 2 },
fuzzy: 0.2,
prefix: true,
},
});

miniSearch.addAll(searchIndex.documents.map((doc, id) => ({ id, ...doc })));

app.get("/docs/search", async (c) => {
const query = c.req.query("q");
const results = miniSearch.search(query, { limit: 10 });

return c.html(`

${results
.map(
(r) => `

• 
  ${r.title}
  ${Math.round(r.score)}
  

`,
)
.join("")}

`);
});
```

**Deployment Requirements**:

- Bundle `minisearch` (~8KB)
- Include `search-index.json`
- Memory: ~1.5x index size

### 1.3 FlexSearch (Maximum Speed)

**Pros**: Fastest in-memory search, language support
**Cons**: More complex API, larger memory footprint

```typescript
import FlexSearch from "flexsearch";
import searchIndex from "./search-index.json";

const index = new FlexSearch.Document({
document: {
id: "path",
index: ["title", "body"],
store: ["title", "path"],
},
tokenize: "forward",
language: "en",
});

searchIndex.documents.forEach((doc) => index.add(doc));

app.get("/docs/search", async (c) => {
const query = c.req.query("q");
const results = index.search(query, { limit: 10, enrich: true });

// FlexSearch returns results per field, merge them
const merged = results.flatMap((r) => r.result);

return c.html(`

${merged
.map(
(r) => `

• ${r.doc.title}

`,
)
.join("")}

`);
});
```

---

## 2. Static Site Search: Pagefind

Pagefind generates a WASM-powered index at build time. Search happens client-side but uses minimal bandwidth.

**Pros**: No server needed, incremental loading, large site support
**Cons**: Client-side (not pure server), initial WASM load (~40KB)

### Build Integration

```bash
# After mdbook-htmx build
npx pagefind --site book/htmx/pages --output-path book/htmx/pagefind
```

### Configuration

```toml
# book.toml
[output.htmx.search]
engine = "pagefind" # Use Pagefind instead of JSON
exclude = ["partials/*", "404.html"]
```

### Client Integration

```html

new PagefindUI({ element: "#search", showImages: false });

```

### Hybrid: Pagefind with HTMX

For server-rendered results with Pagefind's index:

```typescript
// Server loads Pagefind WASM
import { Pagefind } from "@nicco.io/pagefind-preact"; // Or custom loader

app.get("/docs/search", async (c) => {
const query = c.req.query("q");
const results = await pagefind.search(query);
const data = await Promise.all(results.slice(0, 10).map((r) => r.data()));

return c.html(`

${data
.map(
(r) => `

• 
  ${r.meta.title}
  

  ${r.excerpt}

  
  

`,
)
.join("")}

`);
});
```

---

## 3. External Search Engines

For large books or advanced features (facets, typo tolerance, analytics).

### 3.1 Meilisearch (Recommended Self-Hosted)

**Pros**: Fast, typo-tolerant, easy to deploy, Rust-based
**Cons**: Requires separate service

#### Secure Self-Hosting via Cloudflare Tunnel (Recommended)

For production deployments, **never expose Meilisearch directly to the internet**. Instead, use Cloudflare Workers as a proxy layer with Cloudflare Tunnel for private access.

```
Browser (HTMX)
↓ POST /search
Cloudflare Worker (Policy Enforcement)
↓ CF-Access headers
Cloudflare Tunnel (cloudflared)
↓ localhost:7700
Private Meilisearch (LAN/VPC)
```

**Benefits over direct exposure**:

| Risk (Direct Exposure) | Mitigation (Worker Proxy) |
|------------------------|---------------------------|
| Index enumeration | Fixed index, no discovery |
| Filter abuse | Whitelisted filters only |
| Query DoS | Rate limiting, query constraints |
| API key leakage | Keys server-side only |

**Worker acts as policy enforcement layer**:
- Validates authentication before searching
- Sanitizes query input (length limits, character filtering)
- Enforces scope/authorization filters
- Returns HTML partials (not JSON) for HTMX
- Caches popular queries in KV

See **[`examples/meilisearch-cf-tunnel.md`](../examples/meilisearch-cf-tunnel.md)** for complete implementation including:
- Cloudflare Tunnel configuration
- Worker TypeScript with Hono
- Service Token authentication
- Build-time indexing script
- HTMX integration patterns

#### Docker Compose Deployment

```yaml
# docker-compose.yml
version: "3.8"
services:
docs:
build: .
ports:
- "8080:8080"
environment:
- MEILI_URL=http://meilisearch:7700
- MEILI_MASTER_KEY=${MEILI_MASTER_KEY}
depends_on:
- meilisearch

meilisearch:
image: getmeili/meilisearch:v1.6
ports:
- "7700:7700"
environment:
- MEILI_MASTER_KEY=${MEILI_MASTER_KEY}
volumes:
- meili_data:/meili_data

volumes:
meili_data:
```

#### Kubernetes Deployment

```yaml
# meilisearch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: meilisearch
spec:
replicas: 1
selector:
matchLabels:
app: meilisearch
template:
metadata:
labels:
app: meilisearch
spec:
containers:
- name: meilisearch
image: getmeili/meilisearch:v1.6
ports:
- containerPort: 7700
env:
- name: MEILI_MASTER_KEY
valueFrom:
secretKeyRef:
name: meilisearch-secrets
key: master-key
volumeMounts:
- name: data
mountPath: /meili_data
volumes:
- name: data
persistentVolumeClaim:
claimName: meilisearch-pvc
---
apiVersion: v1
kind: Service
metadata:
name: meilisearch
spec:
selector:
app: meilisearch
ports:
- port: 7700
targetPort: 7700
```

#### Index Population (Build Step)

```typescript
// scripts/index-search.ts
import { MeiliSearch } from "meilisearch";
import searchIndex from "../book/htmx/search-index.json";

const client = new MeiliSearch({
host: process.env.MEILI_URL,
apiKey: process.env.MEILI_MASTER_KEY,
});

async function indexDocuments() {
const index = client.index("docs");

// Configure index
await index.updateSettings({
searchableAttributes: ["title", "body", "headings.text"],
displayedAttributes: ["title", "path", "body"],
rankingRules: [
"words",
"typo",
"proximity",
"attribute",
"sort",
"exactness",
],
});

// Add documents
await index.addDocuments(
searchIndex.documents.map((doc, id) => ({ id, ...doc })),
);

console.log("Indexed", searchIndex.documents.length, "documents");
}

indexDocuments();
```

#### Server Integration

```typescript
import { MeiliSearch } from "meilisearch";

const client = new MeiliSearch({
host: process.env.MEILI_URL,
apiKey: process.env.MEILI_SEARCH_KEY, // Read-only key
});

app.get("/docs/search", async (c) => {
const query = c.req.query("q");

const results = await client.index("docs").search(query, {
limit: 10,
attributesToHighlight: ["title", "body"],
highlightPreTag: "",
highlightPostTag: "",
attributesToCrop: ["body"],
cropLength: 50,
});

return c.html(`

${results.hits
.map(
(hit) => `

• 
  
  ${hit._formatted.title}
  
  

  ${hit._formatted.body}

  
  

`,
)
.join("")}

`);
});
```

### 3.2 Typesense

**Pros**: Fastest external engine, C++ based, great typo tolerance
**Cons**: Less cloud availability than Algolia

#### Docker Compose

```yaml
services:
typesense:
image: typesense/typesense:0.25.2
ports:
- "8108:8108"
environment:
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
- TYPESENSE_DATA_DIR=/data
volumes:
- typesense_data:/data

volumes:
typesense_data:
```

#### Server Integration

```typescript
import Typesense from "typesense";

const client = new Typesense.Client({
nodes: [{ host: "typesense", port: 8108, protocol: "http" }],
apiKey: process.env.TYPESENSE_SEARCH_KEY,
});

app.get("/docs/search", async (c) => {
const query = c.req.query("q");

const results = await client.collections("docs").documents().search({
q: query,
query_by: "title,body",
highlight_full_fields: "title,body",
per_page: 10,
});

return c.html(`

${results.hits
.map(
(hit) => `

• 
  ${hit.highlights[0]?.snippet || hit.document.title}
  

`,
)
.join("")}

`);
});
```

### 3.3 Algolia (Managed SaaS)

**Pros**: Zero ops, global CDN, excellent UI components
**Cons**: Pricing at scale, vendor lock-in

```typescript
import algoliasearch from "algoliasearch";

const client = algoliasearch(
process.env.ALGOLIA_APP_ID,
process.env.ALGOLIA_SEARCH_KEY,
);
const index = client.initIndex("docs");

app.get("/docs/search", async (c) => {
const query = c.req.query("q");

const { hits } = await index.search(query, {
hitsPerPage: 10,
attributesToHighlight: ["title", "body"],
});

return c.html(`

${hits
.map(
(hit) => `

• 
  ${hit._highlightResult.title.value}
  

`,
)
.join("")}

`);
});
```

---

## 4. Deployment Architecture

### 4.1 Cloudflare Workers (In-Memory, Simple Sites)

```asciidoc
┌─────────────┐ ┌─────────────────────────────────┐
│ Browser │────▶│ Cloudflare Worker │
│ │ │ ┌───────────────────────────┐ │
│ │ │ │ Fuse.js + search-index │ │
│ │ │ │ (in-memory, <50MB) │ │
│ │◀────│ └───────────────────────────┘ │
└─────────────┘ └─────────────────────────────────┘
```

**Limits**:

- Worker memory: 128MB
- Index must fit in bundled assets
- Best for <1000 pages

### 4.2 Cloudflare Workers + Tunnel (Recommended for Self-Hosted Search)

For larger sites or advanced search features, use Workers as a **policy enforcement proxy** to self-hosted Meilisearch via Cloudflare Tunnel.

```asciidoc
┌─────────────┐ ┌─────────────────────────────────┐
│ Browser │────▶│ Cloudflare Worker │
│ (HTMX) │ │ ┌───────────────────────────┐ │
│ │ │ │ Policy Enforcement │ │
│ │ │ │ - Auth validation │ │
│ │ │ │ - Query sanitization │ │
│ │ │ │ - Rate limiting │ │
│ │ │ └───────────┬───────────────┘ │
│ │ │ │ │
│ │ └──────────────┼──────────────────┘
│ │ │ CF-Access headers
│ │ ┌──────────────▼──────────────────┐
│ │ │ Cloudflare Tunnel │
│ │ │ (cloudflared) │
│ │ └──────────────┬──────────────────┘
│ │ │
│ │ ┌──────────────▼──────────────────┐
│ │ │ Private Meilisearch │
│ │◀────│ (localhost:7700) │
└─────────────┘ └─────────────────────────────────┘
```

**Benefits**:

- Meilisearch never internet-facing
- Service Token authentication via CF Access
- Worker returns HTML partials for HTMX
- Cache layer in Worker KV
- Best for 1000+ pages with advanced features

**Performance**: ~30-50ms E2E (5-20ms Worker→Tunnel, <10ms Meilisearch)

See **[`examples/meilisearch-cf-tunnel.md`](../examples/meilisearch-cf-tunnel.md)** for complete implementation

### 4.3 Docker Compose (Self-Hosted)

```asciidoc
┌─────────────┐ ┌─────────────────────────────────┐
│ Browser │────▶│ Reverse Proxy │
│ │ │ (Caddy/Nginx) │
│ │ └────────────┬────────────────────┘
│ │ │
│ │ ┌────────────┴────────────────────┐
│ │ │ │
│ │ ▼ ▼
│ │ ┌──────────────┐ ┌───────────────────┐
│ │ │ Docs Server │──────────│ Meilisearch │
│ │ │ (Hono/Deno) │ │ (Port 7700) │
│ │ └──────────────┘ └───────────────────┘
└─────────────┘
```

```yaml
# docker-compose.yml
version: "3.8"
services:
caddy:
image: caddy:2-alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
- caddy_data:/data
depends_on:
- docs

docs:
build: .
environment:
- MEILI_URL=http://meilisearch:7700
- MEILI_SEARCH_KEY=${MEILI_SEARCH_KEY}

meilisearch:
image: getmeili/meilisearch:v1.6
environment:
- MEILI_MASTER_KEY=${MEILI_MASTER_KEY}
volumes:
- meili_data:/meili_data

volumes:
caddy_data:
meili_data:
```

### 4.4 Kubernetes (Production)

```asciidoc
┌─────────────┐ ┌─────────────────────────────────┐
│ Browser │────▶│ Ingress Controller │
│ │ │ (nginx-ingress) │
│ │ └────────────┬────────────────────┘
│ │ │
│ │ ┌────────────┴────────────────────┐
│ │ │ Service Mesh │
│ │ │ (optional) │
│ │ └────────────┬────────────────────┘
│ │ │
│ │ ┌────────────┴────────────────────┐
│ │ │ │
│ │ ▼ ▼
│ │ ┌──────────────┐ ┌───────────────────┐
│ │ │ Docs │──────────│ Meilisearch │
│ │ │ Deployment │ │ StatefulSet │
│ │ │ (3 replicas)│ │ (1 replica) │
│ │ └──────────────┘ └───────────────────┘
└─────────────┘ │
┌────────┴────────┐
│ PVC (10Gi) │
└─────────────────┘
```

Full Kubernetes manifests:

```yaml
# docs-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: docs
spec:
replicas: 3
selector:
matchLabels:
app: docs
template:
metadata:
labels:
app: docs
spec:
containers:
- name: docs
image: myregistry/docs-server:latest
ports:
- containerPort: 8080
env:
- name: MEILI_URL
value: "http://meilisearch:7700"
- name: MEILI_SEARCH_KEY
valueFrom:
secretKeyRef:
name: search-secrets
key: meili-search-key
resources:
requests:
memory: "128Mi"
cpu: "100m"
limits:
memory: "256Mi"
cpu: "500m"
---
apiVersion: v1
kind: Service
metadata:
name: docs
spec:
selector:
app: docs
ports:
- port: 80
targetPort: 8080
---
# meilisearch-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: meilisearch
spec:
serviceName: meilisearch
replicas: 1
selector:
matchLabels:
app: meilisearch
template:
metadata:
labels:
app: meilisearch
spec:
containers:
- name: meilisearch
image: getmeili/meilisearch:v1.6
ports:
- containerPort: 7700
env:
- name: MEILI_MASTER_KEY
valueFrom:
secretKeyRef:
name: search-secrets
key: meili-master-key
volumeMounts:
- name: data
mountPath: /meili_data
resources:
requests:
memory: "512Mi"
cpu: "250m"
limits:
memory: "2Gi"
cpu: "1000m"
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
name: meilisearch
spec:
selector:
app: meilisearch
ports:
- port: 7700
targetPort: 7700
clusterIP: None # Headless for StatefulSet
```

---

## 5. Decision Matrix

### Choose Based on Book Size

| Book Size | Pages | Recommended | Alternative |
| ---------- | -------- | ----------- | ------------- |
| Small | <100 | Fuse.js | MiniSearch |
| Medium | 100-500 | MiniSearch | Meilisearch |
| Large | 500-5000 | Meilisearch | Typesense |
| Very Large | >5000 | Typesense | Elasticsearch |

### Choose Based on Deployment

| Deployment | Recommended | Notes |
| -------------------------------- | --------------------- | ------------------- |
| Cloudflare Workers | Fuse.js/MiniSearch | In-memory, bundled |
| Static hosting (Netlify, Vercel) | Pagefind | Client-side WASM |
| Docker Compose | Meilisearch | Easy setup |
| Kubernetes | Meilisearch/Typesense | StatefulSet pattern |
| Enterprise | Algolia | Managed, compliant |

### Choose Based on Features

| Feature | Fuse.js | MiniSearch | Pagefind | Meilisearch | Typesense | Algolia |
| --------------- | ------- | ---------- | -------- | ----------- | --------- | ------- |
| Typo tolerance | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ |
| Prefix matching | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
| Faceted search | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |
| Highlighting | ⚠️ | ⚠️ | ✅ | ✅ | ✅ | ✅ |
| Multi-language | ❌ | ❌ | ✅ | ✅ | ✅ | ✅ |
| Authorization | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ |
| Analytics | ❌ | ❌ | ❌ | ⚠️ | ⚠️ | ✅ |

Legend: ✅ Built-in | ⚠️ Partial/Manual | ❌ Not supported

---

## 6. Authorization-Aware Search

For sites with access control, filter search results based on user permissions.

### Manifest Integration

```json
{
"documents": [
{
"path": "/docs/public-page",
"auth": { "access": "public" }
},
{
"path": "/docs/admin-page",
"auth": { "access": "roles", "roles": ["admin"] }
}
]
}
```

### Server-Side Filtering

```typescript
app.get("/docs/search", async (c) => {
const query = c.req.query("q");
const user = c.get("user");

let results = fuse.search(query);

// Filter by authorization
results = results.filter((r) => {
const page = r.item;
if (page.auth.access === "public") return true;
if (page.auth.access === "authenticated") return !!user;
if (page.auth.access === "roles") {
return page.auth.roles.some((role) => user?.roles.includes(role));
}
return false;
});

return c.html(renderResults(results.slice(0, 10)));
});
```

---

## 7. Configuration Reference

```toml
[output.htmx.search]
enabled = true

# Index format: "json" | "sqlite" | "pagefind"
index-format = "json"

# What to index
heading-split-level = 3 # Index up to H3
include-body = true # Full text search
max-body-length = 10000 # Truncate long documents

# Exclude patterns
exclude = ["drafts/*", "internal/*"]

# External engine (if using Meilisearch/Typesense/Algolia)
[output.htmx.search.external]
engine = "meilisearch" # meilisearch | typesense | algolia
index-name = "docs"
# Credentials in environment variables:
# MEILI_URL, MEILI_MASTER_KEY, MEILI_SEARCH_KEY
```

---

## References

- [Fuse.js Documentation](https://fusejs.io/)
- [MiniSearch Documentation](https://lucaong.github.io/minisearch/)
- [Pagefind Documentation](https://pagefind.app/)
- [Meilisearch Documentation](https://docs.meilisearch.com/)
- [Typesense Documentation](https://typesense.org/docs/)
- [Algolia Documentation](https://www.algolia.com/doc/)