📋 Fix Excessive Cloning in Search Pipelines

📋 Fix Excessive Cloning in Search Pipelines

Issue Description

Performance analysis identified excessive memory allocations and cloning operations throughout the search and document processing pipelines. The current implementation frequently clones large data structures, causing significant memory pressure and reducing overall performance.

📍 Problem Areas

Primary Files:

• crates/terraphim_service/src/lib.rs - Document processing with heavy cloning
• crates/terraphim_middleware/src/haystack/mod.rs - Haystack processing cloning large documents
• crates/terraphim_automata/src/autocomplete.rs - Text processing with unnecessary copies
• crates/terraphim_service/src/score/mod.rs - Scoring pipeline cloning search results

Specific Issues:

1. Large document cloning: Full document structures copied for each search operation
2. String allocations: Excessive String::clone() operations in text processing
3. Vector cloning: Large result vectors cloned unnecessarily
4. Zero-copy opportunities: Missing opportunities for reference-based operations

🎯 Solution Approach

Phase 1: Implement Zero-Copy Text Processing

// Current problematic pattern:
pub struct Document {
    id: String,
    title: String,
    body: String,
    url: String,
}

pub fn search_documents(documents: Vec<Document>, query: &str) -> Vec<SearchResult> {
    documents.into_iter()
        .filter(|doc| doc.body.contains(query)) // Clones entire body
        .map(|doc| SearchResult {
            id: doc.id.clone(),           // Unnecessary clone
            title: doc.title.clone(),     // Unnecessary clone
            snippet: extract_snippet(&doc.body, query),
        })
        .collect()
}

// Optimized pattern with references:
pub struct Document {
    id: String,
    title: String,
    body: String,
    url: String,
}

pub struct SearchResult<'a> {
    id: &'a str,
    title: &'a str,
    snippet: String,
    document: &'a Document,
}

pub fn search_documents<'a>(
    documents: &'a [Document], 
    query: &str
) -> Vec<SearchResult<'a>> {
    documents.iter()
        .filter(|doc| doc.body.contains(query))
        .map(|doc| SearchResult {
            id: &doc.id,
            title: &doc.title,
            snippet: extract_snippet(&doc.body, query),
            document: doc,
        })
        .collect()
}

Phase 2: Use Cow for Conditional Cloning

use std::borrow::Cow;

pub struct ProcessedText<'a> {
    original: Cow<'a, str>,
    processed: Option<String>,
}

impl<'a> ProcessedText<'a> {
    pub fn new(text: &'a str) -> Self {
        Self {
            original: Cow::Borrowed(text),
            processed: None,
        }
    }
    
    pub fn get_processed(&mut self) -> &str {
        if self.processed.is_none() {
            self.processed = Some(self.process_text(&self.original));
        }
        self.processed.as_ref().unwrap()
    }
    
    fn process_text(&self, text: &str) -> String {
        // Text processing logic that requires allocation
        text.to_lowercase()
    }
}

Phase 3: Implement Reference-Based Search Results

// Instead of cloning large structures, use references:
#[derive(Clone)]
pub struct SearchResultRef {
    document_id: u64,
    relevance_score: f32,
    match_positions: Vec<TextMatch>,
}

pub struct SearchEngine {
    documents: Arc<Vec<Document>>,
    index: SearchIndex,
}

impl SearchEngine {
    pub fn search(&self, query: &str) -> Vec<SearchResult> {
        let matching_docs = self.index.find_matching_docs(query);
        
        matching_docs.into_iter()
            .map(|doc_ref| SearchResult {
                document: &self.documents[doc_ref.id],
                score: doc_ref.score,
                matches: doc_ref.matches,
            })
            .collect()
    }
}

Phase 4: Optimize String Operations

// Use string slices and references where possible:
pub fn extract_snippet_optimized(text: &str, query: &str, context: usize) -> &str {
    if let Some(pos) = text.find(query) {
        let start = pos.saturating_sub(context);
        let end = (pos + query.len() + context).min(text.len());
        &text[start..end]
    } else {
        &text[..text.len().min(200)]
    }
}

// Use small string optimization for common cases:
use smallvec::SmallVec;
use std::borrow::Cow;

pub type TextBuffer = SmallVec<[u8; 64]>;

pub struct TokenizedText {
    tokens: Vec<Cow<'static, str>>,
    original_len: usize,
}

📋 Implementation Tasks

1. Audit cloning patterns:

   • Identify all .clone() calls in search pipelines
   • Analyze memory allocation patterns
   • Profile memory usage during search operations

2. Implement zero-copy patterns:

   • Replace owned strings with references where possible
   • Use Cow<str> for conditional cloning
   • Implement reference-based search results

3. Optimize data structures:

   • Use SmallVec for small collections
   • Implement string interning for repeated strings
   • Add compact storage for frequently accessed data

4. Reduce allocations in hot paths:

   • Pre-allocate buffers for repeated operations
   • Use object pools for frequently created objects
   • Implement lazy evaluation for expensive operations

5. Add memory profiling:

   • Add memory usage tracking
   • Implement allocation count monitoring
   • Add benchmark tests for memory efficiency

📊 Expected Performance Improvements

• Memory usage: 30-40% reduction through fewer allocations
• Search latency: 20-30% faster due to reduced copying
• CPU usage: 15-25% reduction from fewer allocation/deallocation cycles
• Cache efficiency: Better cache locality with reference-based access

🧪 Testing Strategy

Unit Tests

• Test reference-based search result handling
• Verify correctness of zero-copy text processing
• Test memory management under various loads

Integration Tests

• End-to-end search with memory profiling
• Concurrent search operation memory tests
• Large document processing validation

Benchmarks

• Memory allocation rate benchmarks
• Search performance with large document sets
• Cache hit/miss rate measurements

🔧 Specific Optimizations

Document Storage Optimization

// Store documents once, reference them multiple times:
pub struct DocumentStore {
    documents: Vec<Document>,
    id_to_index: HashMap<u64, usize>,
    // Add compression for large documents
    compressed_bodies: Option<Vec<CompressedData>>,
}

Search Result Optimization

// Use compact representation for search results:
#[repr(C)]
pub struct CompactSearchResult {
    document_id: u64,
    score: f32,
    match_count: u32,
    // Variable length data stored separately
}

String Interning

use string_interner::StringInterner;

pub struct InternedStrings {
    interner: StringInterner<usize>,
    // Cache frequently accessed strings
    common_strings: HashMap<String, usize>,
}

📚 Resources

• Rust Ownership and Borrowing
• Zero-Copy Programming in Rust
• Cow Documentation
• SmallVec Crate

🏁 Definition of Done

• Excessive cloning is eliminated from search pipelines
• Memory profiling shows 30%+ reduction
• All existing tests pass
• New memory efficiency tests are added
• Documentation is updated
• Code review is approved

Related to: #193 (Performance Optimization Epic), #194 (Async Blocking Operations)
Estimated Effort: 3-4 days
Priority: High (Memory efficiency is critical)

Start by auditing clone and allocation patterns in crates/terraphim_service/src/lib.rs, crates/terraphim_middleware/src/haystack/mod.rs, crates/terraphim_automata/src/autocomplete.rs, and crates/terraphim_service/src/score/mod.rs. Review existing search tests and benchmarks before choosing a bounded optimization; done means reduced cloning, added memory-efficiency coverage, and all existing tests passing.