[QUESTION]

### Your Question

## Replication of the Contextual Retrieval cookbook

I replicated all four pipelines from the cookbook on the released codebase dataset
(9 codebases, 737 chunks, 248 queries) using the same stack: voyage-2 embeddings,
claude-haiku for context generation, Elasticsearch BM25, Cohere reranker.

My results:

| Pipeline | Pass@5 | Pass@10 | Pass@20 |
|---|---|---|---|
| Basic RAG | 80.92% | 87.15% | 90.06% |
| Contextual Embeddings | 88.42% | 93.28% | 95.13% |
| Contextual BM25 | 88.89% | 93.45% | 96.07% |
| Reranking | 93.06% | 95.93% | 98.35% |

Results are consistent with the cookbook. The biggest single gain is from contextual
embeddings, with the reranker adding a further meaningful improvement on top.

---

## Hybrid + Reranking

The blog post diagram shows the final best-performing system as **contextual hybrid
search + reranker** (BM25 + dense embeddings fused via weighted RRF, then reranked).
This pipeline is described in the blog but not implemented in the cookbook, which
stops at dense + reranker.

I implemented it:

| Pipeline | Pass@5 | Pass@10 | Pass@20 |
|---|---|---|---|
| Reranking | 93.06% | 95.93% | **98.35%** |
| Hybrid + Reranking | **93.46%** | 95.93% | 97.95% |

On the codebase dataset, Hybrid + Reranking outperforms Reranking at Pass@5
(+0.40 pp), ties at Pass@10, and slightly underperforms at Pass@20 (−0.40 pp).
In other words, on this dataset the hybrid pipeline is not clearly the best
configuration, depending on the cut-off used.

---

## Two questions

**1.** Did you observe the same pattern on the codebase dataset — i.e. that
hybrid + reranking does not clearly reduce failure rate at top-20 compared to
dense + reranking alone? Or does this crossover disappear with a different
configuration (e.g. larger candidate pool before reranking, different RRF weights)?

**2.** The blog reports results across four domains (codebases, fiction, ArXiv
papers, science papers) but only the codebase dataset is publicly available.
Would it be possible to release the other domain datasets and per-domain results,
or at minimum share the per-domain breakdown? This would make the cookbook results
independently verifiable and would help practitioners understand when the hybrid
approach is worth the added infrastructure cost.

Happy to share my full replication code and experiment logs if useful.

### Related Notebook (if applicable)

contextual-embeddings/guide.ipynb

### What I've Tried

_No response_

### Additional Context

_No response_