[Tracker] `compose_index` design for multi-index querying

Thank you again for the valuable discussion in PR #713. This issue summarizes key points raised by @cjnolet, @achirkin, and @rhdong regarding the current compose_index design, and proposes a space for continued discussion on generalizing and improving it.

Context

The compose_index implementation in the PR enables logical merging of multiple CAGRA indices for unified search. This is useful in scenarios like Lucene-style segment-based indexing, where multiple small indices are produced and need to be queried as a whole without physically rebuilding them.

While the feature was merged to support immediate use cases, several important architectural questions were raised that merit further attention.

Summary of Discussion

1. Algorithm-specific vs general abstraction

• @cjnolet noted that limiting composition to CAGRA is overly restrictive. The ability to logically merge indices is generally useful and should ideally be index-agnostic.
• @achirkin agreed and felt the current API was artificially scoped to a single algorithm.

2. Standalone implementation vs reusing existing mechanisms

• Both @achirkin and @cjnolet observed that cuVS already has mechanisms for managing multiple indices, such as dynamic batching and SNMG multi-GPU wrappers. These systems could potentially be reused to support logical composition more broadly.
• They advocated for leveraging these rather than duplicating functionality under a new composite_index path.

3. Why a separate composite_index was introduced

• @rhdong explained that he initially attempted to convert cuvs::neighbors::index into an abstract base class, allowing composite_index to inherit from it. However, due to widespread use of the concrete class and time constraints, this proved impractical.
• As a result, a CAGRA-specific workaround was implemented to deliver value in the short term, with the understanding that future refactoring is welcome and necessary.

4. Concerns about overhead (NCCL, locks, etc.)

• @rhdong expressed concern that adapting the SNMG-style implementation might introduce unnecessary complexity (e.g., NCCL communication, locking) for single-GPU logical merges.
• @achirkin suggested that a lightweight adjustment of the current batching logic might be simpler and more efficient than the newly added interface.

5. API location and layering

• @achirkin and @cjnolet both suggested (and @rhdong agrees) that a general-purpose compose_index utility should live outside the cagra module and exist at a higher, shared layer of abstraction.
• This would make it easier to apply the same logic to other index types and allow full integration with APIs like search, search_with_filtering, and search_snmg.
• @rhdong suggested moving the compose_index to the upper layer, since it already supports different index types.

6. Short-term vs long-term tradeoff

• There was consensus that supporting the current use case promptly was important, but longer-term improvements should aim to make logical composition a first-class concept within cuVS—alongside batching and multi-GPU support.
• @cjnolet proposed using a follow-up issue to track that evolution, hence this post.

Need to clarify

• If @cjnolet @achirkin are suggesting to unify the multi-index logic in dynamic batching, SNMG, and CAGRA logical merge to one, or just only want a compose_index for merge in different algorithms?

Next Steps

This issue can serve as a space to:

• Explore a generic composite_index interface design
• Evaluate how dynamic batching and SNMG can be reused for this purpose
• Discuss options to balance generality and overhead, especially for single-GPU setups