LLMQuant / LLMQuant/quant-mind

feat: page-preserving TreeKnowledge build + agentic navigation

Open
#95 0 comments 0 reactions 1 assignee View on GitHub

@keli-wen is already working on this.

Since Jul 10, 2026.

area: mind area: preprocess type: feature
Dominant language
Python
Stars
3k
Forks
484
PR merge metrics
No merged PRs in 30d

Description

## Feature Summary

Add page-preserving tree construction and agentic navigation for supported financial long documents: papers, filings, transcripts, and reports.

This complements semantic search rather than replacing it:

- semantic search finds candidates across a collection;
- agentic navigation reasons through one candidate document's hierarchy.

The semantic baseline in #111 and its paper integration in #112 land first.

## Motivation

Long financial documents do not fit in one extraction pass, and flat chunks lose document structure and page provenance. A page-anchored `TreeKnowledge` provides:

1. structure-aware navigation through titles and summaries;
2. page-anchored evidence through canonical citations;
3. one shared tree contract across supported financial document types.

VectifyAI/PageIndex validates the approach, but implementation choice remains open. Reuse or adapt low-level PageIndex mechanics where useful while preserving QuantMind's own tree, time, provenance, and citation contracts.

## Ownership

- `preprocess/`: preserve page boundaries and deterministic outline signals.
- `knowledge/`: reuse `TreeKnowledge`, `TreeNode`, and `Citation`; do not add a second `PageIndexNode` tree model. Extend `Citation` with optional `end_page`: `page` is the 1-based inclusive start, an omitted end means one page, and a present end must satisfy `end_page >= page`.
- `flows/`: build/enrich canonical trees and summaries with the Agents SDK.
- `library/`: persist canonical trees and provide semantic candidates through #111; this issue adds no second store or semantic index.
- `mind/navigation/`: perform agentic traversal over titles and summaries.
- `flows/` or an agent application: optionally compose semantic shortlist and tree navigation after both primitives exist.

## API Direction

```python
from quantmind.flows import extract_paper_knowledge
from quantmind.mind.navigation import navigate_tree
from quantmind.preprocess import build_page_outline

outline = build_page_outline(pages)
tree = await extract_paper_knowledge(outline, cfg=cfg)
hits = await navigate_tree(tree, question)
```

Per-domain public extraction names must follow `docs/design/en/operations.md`; shared enrichment helpers remain private. Results identify canonical item/node IDs and page evidence; they do not expose a provider-specific response type.

## Acceptance Criteria

- [ ] Runs after the semantic baseline in #111 is available for comparison.
- [ ] Preserves 1-based inclusive page spans from supported parsed inputs and validates `page <= end_page` when an end is present.
- [ ] Covers citation page-span serialization round trips.
- [ ] Builds a valid canonical `TreeKnowledge` without a second tree schema.
- [ ] Keeps deterministic page/outline work free of LLM calls.
- [ ] Keeps summary enrichment in `flows/` and navigation in `mind/navigation/`.
- [ ] Depends on #71 for a `mind` scaffold and import-linter boundary. If that scaffold has not landed, this issue adds the minimal package and enforces `mind -> library -> knowledge` without persistence/index ownership in `mind`.
- [ ] Returns canonical node IDs plus page-anchored evidence.
- [ ] Reuses `LocalKnowledgeLibrary` for persistence or semantic shortlist when needed.
- [ ] Includes offline tests and one focused example.
- [ ] Compares navigation quality with the semantic baseline on a bounded financial-document fixture.
- [ ] `scripts/verify.sh` passes.

## Non-goals

- A general-purpose RAG or PageIndex framework
- A second `PageIndexNode` or generic `Document` schema
- A second persistence or semantic-index layer
- Knowledge Graph construction
- A provider registry or retriever hierarchy
- Answer synthesis in the navigation primitive

## Related Issues

- Depends on #111 for semantic retrieval and canonical library persistence.
- Follows #112 for the first paper extraction-to-search baseline.
- Depends on #71 for the `mind` cognitive boundary.

## Implementation Considerations

- Functions over class hierarchies; no generic retriever abstraction.
- Standard parsing without OCR is acceptable for V1.
- The implementation may reuse PageIndex code or concepts subject to license, dependency, and contract review; do not commit to reimplementing commodity mechanics before that review.

### Breaking Changes

- [ ] This feature introduces breaking changes.
- [x] This feature is backward compatible.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.