[Mixed-source orchestration] Phase 4: Support cross-format Compare with native source engines

@paullizer is already working on this.

Since Jul 22, 2026.

Parent: #1055

# Phase 4 - Cross-Format Compare

Planning baseline version: **0.250.061**

Status: **Proposed**

Depends on: Phase 1; Phase 3 evidence contracts are strongly recommended

## Objective

Support one-left-to-many-right comparison across any supported source combination:

- table to table
- narrative to table
- table to narrative
- narrative to narrative

Every source must be summarized with its native engine before pairwise comparison.

## Current Defects Addressed

- Mixed Compare abandons tabular tooling when any selected source is non-tabular.
- The tabular comparison helper only succeeds when every comparison source is tabular.
- The generic comparison path summarizes CSV/XLSX through ordinary document chunks.
- There is no engine-neutral comparison evidence contract.

## Comparison Strategy

### Narrative to narrative

Preserve current document analysis summaries and pairwise reduction.

### Tabular to tabular

For each source:

- schema and sheet coverage
- relevant counts, totals, distributions, keys, dates, statuses, anomalies
- query-specific computed evidence

When the request requires row-level alignment, joins, overlap, or deltas, invoke validated multi-file tabular operations rather than comparing prose summaries alone.

### Narrative to tabular

Produce:

- narrative evidence summary from document analysis
- computed tabular evidence from tabular tools
- one comparison prompt that labels evidence type and source role

Do not imply row-level support from a narrative document unless citations provide it.

## Scope

### 1. Comparison manifest

Resolve Source and Targets through the Phase 1 manifest. Preserve role labels and request order.

### 2. Native summary pass

Replace the current universal `run_document_analysis(...)` summary pass with per-source engine dispatch.

Each result becomes a comparison-ready evidence envelope containing:

- source role
- concise summary
- structured facts
- citations
- coverage/failure state
- source version

### 3. Pairwise comparison

Reuse the existing left-to-many sequence, but build each pairwise prompt from engine-neutral envelopes.

For table-to-table pairs, allow a tabular comparison plan to add structured operations before synthesis.

### 4. Reduction

Preserve the current multi-target final reduction. It must state:

- which targets were compared
- which engines supplied evidence
- failed/partial targets
- whether conclusions are row-level, aggregate, or narrative

### 5. Coverage and artifacts

Merge native coverage and generated artifacts without treating an export as comparison prose.

## Acceptance Criteria

- CSV Source vs XLSX Target invokes tabular tooling for both.
- PDF Source vs CSV Target invokes narrative analysis and tabular tooling.
- XLSX Source vs DOCX Target works in the reverse direction.
- One failed Target remains visible while successful Targets are compared.
- File order and duplicate filenames across scopes do not alter source identity.
- Table-to-table overlap/delta questions execute validated cross-table operations.
- Pairwise and final responses cite the correct source documents.
- The response distinguishes aggregate tabular conclusions from narrative claims.

## Tests

- CSV vs XLSX
- PDF vs CSV
- XLSX vs DOCX
- one Source with mixed Target types
- duplicate filenames in different scopes
- cross-table shared-key overlap
- schema mismatch and missing join key
- partial target failure
- source authorization revoked before execution
- source version change
- agent/model and streaming/non-streaming parity
- generated artifacts and token aggregation

## Rollout

Use `enable_cross_format_compare`. Start with one Source and one Target, then enable one-to-many mixed Targets after pairwise coverage is stable.

## Rollback

Disable cross-format support. Same-type comparisons can continue on their established paths; mixed requests should return a clear validation message rather than silently using the wrong engine.

## Exit Gate

Proceed when every source in a mixed comparison is demonstrably processed by its native engine and pairwise/final coverage is complete or explicitly partial.

## Not in This Phase

- Many-to-many comparisons
- Exhaustive All Documents comparison
- Automatic comparison target discovery
- Conversation follow-up source reuse