crate / crate/cratedb-guide

Documentation: Consolidate search documentation to avoid duplication

Open
#400 0 comments 0 reactions 0 assignees View on GitHub
maintenance question
Dominant language
No language data
Stars
3
Forks
4
Avg merge
3d 18h
Merged PRs (30d)
1

Description

## Context

This issue tracks recommendations for consolidating CrateDB search documentation following the addition of new getting-started content in PR #264.

It looks like the idea, layout and structure of the ingredients in this patch have been derived from the existing introductory section about [CrateDB's search features](https://cratedb.com/docs/guide/feature/search/) in one way or another, maybe also its content.

It can be a good alternative to invest into improving the canonical existing pages instead of duplicating the same topic once more again, based on content that has apparently been created using GenAI / LLMs (large language models), so it is blurry and possibly incorrect, at least.

Maybe the most important ideas from the new content can be curated and merged into the existing pages, together with suggestions how to improve their guidance?

Please clarify.

**Related:**
- PR: #264
- Discussion: https://github.com/crate/cratedb-guide/pull/264#discussion_r2296182109

## Current Situation

The repository now has two sets of search documentation:

1. **Existing comprehensive documentation**: `docs/feature/search/` (~1,185 lines)
- Well-structured with subdirectories (fts/, geo/, vector/, hybrid/)
- Detailed technical content with advanced topics
- Includes analyzer configuration, tutorials, best practices

2. **New getting-started content**: `docs/start/query/search/` (~555 lines)
- Simplified introductory content (47% size of existing)
- Flat file structure
- Cleaned up from initial GenAI generation but still has gaps

### Quality Assessment

**Improvements made:**
- Most GenAI slop removed (only 2 "AI-powered" phrases remain)
- Content is cleaner and more focused

**Remaining concerns:**
- **Technical accuracy gaps**: BM25 mentions: 6 in new content vs. 22 in existing
- `fulltext.md` has 0 BM25 references (vs. 16 in `fts/index.md`)
- **Formatting artifacts**: Stray "sqlCopierModifier--" found in `geo.md`
- **Topic duplication**: All four main topics covered in both locations
- **Maintenance burden**: Two documentation sets to maintain

## Recommendations

### 1. Consolidate, Don't Duplicate

- Keep `docs/feature/search/` as the canonical, comprehensive documentation
- Extract genuinely useful quick-start examples from `docs/start/query/search/`
- Integrate them into existing documentation as introductory sections
- Remove duplicate content to maintain single source of truth

### 2. Improve Existing Documentation

- Add "Getting Started" or "Quick Start" sections to each topic in `docs/feature/search/`
- Include simple, practical examples before diving into advanced features
- Improve navigation and cross-linking between getting-started and in-depth content
- Consider restructuring to have beginner-friendly entry points

### 3. Maintain Single Source of Truth

- Revert the toctree change from `search/index` back to `../../feature/search/index` in `docs/start/query/index.md`
- Or create a simple landing page in `docs/start/query/search/` that links to sections within `docs/feature/search/`
- Focus all improvement efforts on the existing, technically superior documentation
- Avoid parallel content that can diverge over time

### 4. Quality Control

- Address technical accuracy gaps (e.g., proper BM25 coverage in fulltext search)
- Fix formatting artifacts (e.g., "sqlCopierModifier--" in geo.md)
- Ensure all examples are tested and accurate
- Review for completeness against the comprehensive existing documentation

## Benefits of Consolidation

- **Reduced maintenance burden**: Single documentation set to update
- **Improved accuracy**: Focus quality efforts on one authoritative source
- **Better user experience**: Clear path from beginner to advanced topics
- **Avoid confusion**: No conflicting or duplicate information

## Implementation Approach

1. Audit both documentation sets for unique, valuable content
2. Identify gaps in existing documentation that new content addresses
3. Create enhancement plan for `docs/feature/search/` with beginner sections
4. Migrate valuable examples and patterns
5. Update navigation and cross-references
6. Remove or redirect duplicate content

---

cc: @surister @kneth

Contributor guide

No contributing guide indexed for this repository

Research direction

Compare docs/feature/search/ with docs/start/query/search/ and inspect docs/start/query/index.md plus the related PR #264 discussion. Start by auditing unique content, BM25 coverage, formatting artifacts, and navigation. Done means useful introductory material is integrated into the canonical documentation, duplicate content is removed or redirected, and cross-references are updated.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.