[BUG] LanceDBStorage.delete() ignores scope/older_than filters and causes accidental mass deletion when combining record_ids with categories

Description

LanceDBStorage.delete() contains several critical filter isolation bugs that lead to accidental mass data deletion, violation of multi-tenant/scoped memory isolation, and ignored timestamp filters.

When inspecting lib/crewai/src/crewai/memory/storage/lancedb_storage.py (lines 422–448):

with store_lock(self._lock_name):
    if record_ids and not (categories or metadata_filter):
        before = int(self._table.count_rows())
        ids_expr = ", ".join(f"'{rid}'" for rid in record_ids)
        self._do_write("delete", f"id IN ({ids_expr})")
        return before - int(self._table.count_rows())
    if categories or metadata_filter:
        rows = self._scan_rows(scope_prefix)
        to_delete: list[str] = []
        for row in rows:
            record = self._row_to_record(row)
            if categories and not any(c in record.categories for c in categories):
                continue
            if metadata_filter and not all(...):
                continue
            if older_than and record.created_at >= older_than:
                continue
            to_delete.append(record.id)  # <--- BUG: record_ids is never checked here


`Key Issues`
- Accidental Mass Data Deletion: When both record_ids and categories (or metadata_filter) are provided (e.g. memory.forget(record_ids=["rec1"], categories=["finance"])), the code enters the second branch where record_ids is completely ignored. It appends every record matching the category to to_delete, deleting all records in that category regardless of their ID
- Multi-Tenant / Scope Isolation Breach: When record_ids is passed without categories, scope_prefix is ignored: delete id IN ('id') runs across the entire database. Calling memory.forget(record_ids=["other_tenant_rec"]) on a scoped Memory(root_scope="/tenantA") will delete records belonging to /tenantB.
- Bypassed older_than Filter: Calling storage.delete(record_ids=["rec1"], older_than=thirty_days_ago) ignores older_than and deletes the record immediately even if it was just created.
- Performance / Memory Overhead: During if categories or metadata_filter, _scan_rows(scope_prefix) fetches all columns including the heavy 3,072-dimensional vector embedding for every row into RAM just to check string metadata.

### Steps to Reproduce

1. Initialize LanceDBStorage.
2. Insert multiple records across two scopes (/tenant1 and /tenant2) and categories (catA and catB).
3. Call storage.delete(scope_prefix="/tenant1", record_ids=["rec2_in_tenant2"]) -> Observe that rec2 in /tenant2 is deleted.
4. Call storage.delete(scope_prefix="/tenant1", record_ids=["rec1"], categories=["catA"]) -> Observe that all records in catA (including rec3 which is not in record_ids) are deleted.

### Expected behavior

1. storage.delete(scope_prefix="/tenant1", record_ids=["rec2"]) should only delete rec2 if it resides within /tenant1 (should delete 0 if rec2 is in /tenant2).
2. storage.delete(scope_prefix="/tenant1", record_ids=["rec1"], categories=["catA"]) should only delete rec1 if it belongs to catA, leaving all other records in catA untouched.
3. older_than should be respected when passed alongside record_ids.

### Screenshots/Code snippets

from datetime import datetime, timezone
from crewai.memory.storage.lancedb_storage import LanceDBStorage
from crewai.memory.types import MemoryRecord

storage = LanceDBStorage(path="/tmp/test_delete_bug", vector_dim=4)
now = datetime.now(timezone.utc)

r1 = MemoryRecord(id="rec1", content="c1", scope="/tenant1", categories=["catA"], created_at=now, embedding=[0.0]*4)
r2 = MemoryRecord(id="rec2", content="c2", scope="/tenant2", categories=["catA"], created_at=now, embedding=[0.0]*4)
r3 = MemoryRecord(id="rec3", content="c3", scope="/tenant1", categories=["catA"], created_at=now, embedding=[0.0]*4)
storage.save([r1, r2, r3])

# Bug 1: Cross-scope breach
deleted = storage.delete(scope_prefix="/tenant1", record_ids=["rec2"])
print("Deleted rec2 with scope=/tenant1:", deleted)
print("rec2 still exists in /tenant2?:", storage.get_record("rec2") is not None)
# Output: Deleted = 1, exists = False (Deleted across scopes!)

# Bug 2: Mass deletion
deleted = storage.delete(scope_prefix="/tenant1", record_ids=["rec1"], categories=["catA"])
print("Deleted count when asking for rec1:", deleted)
print("Was rec3 (NOT in record_ids) deleted?:", storage.get_record("rec3") is None)
# Output: Deleted = 1, rec3 deleted = True (Wiped out unintended record!)

### Operating System

macOS Sonoma

### Python Version

3.12

### crewAI Version

1.15.21

### crewAI Tools Version

1.15.21

### Virtual Environment

Venv

### Evidence

Running the reproduction script demonstrates:

Deleted rec2 with scope=/tenant1: 1
rec2 still exists in /tenant2?: False

Deleted count when asking for rec1: 1
Was rec3 (NOT in record_ids) deleted?: True

The storage layer completely ignores scope_prefix when record_ids is passed, and completely ignores record_ids when categories is passed.

### Possible Solution

1. Fast SQL Path: When neither categories nor metadata_filter are provided, combine record_ids (`id IN (...)`), scope_prefix (`scope LIKE '...' OR scope = '/'`), and older_than (`created_at < '...'`) directly into LanceDB's native SQL where clause.
2. In-Memory Filter Path: When categories or metadata_filter are provided, enforce `allowed_ids = set(record_ids) if record_ids else None` inside the filtering loop so records not in record_ids are never deleted.
3. Performance: Pass `columns=["id", "content", "scope", "categories_str", "metadata_str", "created_at"]` to `_scan_rows()` to avoid reading heavy vector columns into memory during deletion scans.

### Additional context

I have verified this reproduction locally and have the fix and regression unit tests ready to submit as a PR if approved.

Start in lib/crewai/src/crewai/memory/storage/lancedb_storage.py around lines 422–448 and trace both delete branches, including _scan_rows(scope_prefix). Use the reproduction steps in the issue to verify scope, record_ids, categories, and older_than combinations. Done means only records matching every supplied filter are deleted, with scoped records protected and heavy vector columns avoided during filtered scans.