[BUG] LanceDBStorage.list_records() returns oldest records instead of newest first due to premature query truncation
Open
@Rohitkanithi is already working on this.
Since Sep 11, 2026.
bug
- Dominant language
- Python
- Stars
- 58.8k
- Forks
- 8.5k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 109
Description
Description
In LanceDBStorage (lib/crewai/src/crewai/memory/storage/lancedb_storage.py), list_records() is documented to return records ordered by created_at descending:
"List records in a scope, newest first. Returns: List of MemoryRecord, ordered by created_at descending."
However, list_records() currently returns the oldest records in the table rather than the newest.
Root Cause:
In lancedb_storage.py lines 507–510:
rows = self._scan_rows(scope_prefix, limit=limit + offset)
records = [self._row_to_record(r) for r in rows]
records.sort(key=lambda r: r.created_at, reverse=True)
return records[offset : offset + limit]
### Steps to Reproduce
```text
1. Initialize LanceDBStorage in a temporary directory.
2. Insert 10 records with sequential created_at timestamps (rec_0 oldest, rec_9 newest).
3. Call storage.list_records(limit=3).
4. Inspect the returned IDs.
### Expected behavior
list_records(limit=3) should return the 3 most recently created records:
['rec_9', 'rec_8', 'rec_7']
### Screenshots/Code snippets
import tempfile
from datetime import datetime, timezone, timedelta
from crewai.memory.storage.lancedb_storage import LanceDBStorage
from crewai.memory.types import MemoryRecord
with tempfile.TemporaryDirectory() as tmpdir:
storage = LanceDBStorage(path=tmpdir)
base_time = datetime.now(timezone.utc)
# Insert 10 records: rec_0 is oldest, rec_9 is newest
records = [
MemoryRecord(
id=f"rec_{i}",
content=f"content {i}",
created_at=base_time + timedelta(minutes=i),
embedding=[0.1] * 1536
)
for i in range(10)
]
storage.save(records)
# Request the 3 newest records
results = storage.list_records(limit=3)
print("Returned IDs:", [r.id for r in results])
# Expected: ['rec_9', 'rec_8', 'rec_7']
# Actual: ['rec_2', 'rec_1', 'rec_0']
### Operating System
macOS Sonoma
### Python Version
3.12
### crewAI Version
1.15.21
### crewAI Tools Version
1.15.21
### Virtual Environment
Venv
### Evidence
Running the reproduction script outputs:
Returned IDs: ['rec_2', 'rec_1', 'rec_0']
The 3 oldest records (rec_0, rec_1, rec_2) were fetched and sorted descending, while the actual newest records (rec_9, rec_8, rec_7) were truncated at the storage layer and never returned.
### Possible Solution
In `LanceDBStorage.list_records`:
Do not truncate `_scan_rows` before sorting. Scan the candidate rows in the scope (using the default `_SCAN_ROWS_LIMIT` of 50,000, identical to how `get_scope_info` works and how `QdrantEdgeStorage.list_records` works), sort by `created_at` descending, and then apply the pagination slice:
```python
def list_records(
self, scope_prefix: str | None = None, limit: int = 200, offset: int = 0
) -> list[MemoryRecord]:
rows = self._scan_rows(scope_prefix)
records = [self._row_to_record(r) for r in rows]
records.sort(key=lambda r: r.created_at, reverse=True)
return records[offset : offset + limit]
### Additional context
```text
I have verified this reproduction locally and have the fix and regression unit test ready to submit as a PR if approved.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.