LLMQuant / LLMQuant/quant-mind
paper_flow() fails on its own README example: dict-typed nodes field breaks strict structured output, UUID id fields reject the model's natural output
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3k
- Forks
- 484
- PR merge metrics
- No merged PRs in 30d
Description
Summary
The single-paper example in the README does not run against the currently resolved dependency versions (openai-agents>=0.14, no ceiling, resolves to 0.18.1):
import asyncio
from quantmind.configs import PaperFlowCfg
from quantmind.configs.paper import ArxivIdentifier
from quantmind.flows import paper_flow
async def main() -> None:
paper = await paper_flow(
ArxivIdentifier(id="2401.12345"),
cfg=PaperFlowCfg(model="gpt-4o-mini"),
)
asyncio.run(main())
Two separate bugs stack on top of each other.
Bug 1 — Agent(output_type=Paper) raises before any LLM call
TreeKnowledge.nodes is dict[UUID, TreeNode] (quantmind/knowledge/_tree.py). OpenAI's strict-mode structured output — the default in current openai-agents releases — cannot represent dict-typed fields at all (arbitrary keys are incompatible with a fixed JSON Schema). paper_flow (quantmind/flows/paper.py) passes the bare Paper type straight into Agent(output_type=out_type), with no strict_json_schema=False escape hatch, so the run fails immediately:
agents.exceptions.UserError: Strict JSON schema is enabled, but the output type is not valid.
Either make the output type strict, or wrap your type with AgentOutputSchema(YourType, strict_json_schema=False)
Bug 2 — once strict mode is off, UUID-typed id fields reject the model's output
Wrapping the output type in AgentOutputSchema(Paper, strict_json_schema=False) gets past bug 1, but a non-strict schema drops the UUID format constraint. The extraction agent then does exactly what you'd expect an LLM to do when free-form: it fills TreeNode.node_id / parent_id / children_ids with readable slugs ("root", "introduction", "methodology") and BaseKnowledge.id with the paper's arXiv id ("2404.11584"), rather than leaving these at their UUID4 defaults. Validation then fails with ~80+ uuid_parsing errors.
Repro
Ran the unmodified README example against a real paper (arXiv 2404.11584, "The landscape of emerging AI agent architectures for reasoning, planning, and tool calling: A survey", model gpt-4o-mini) on the current main (8e21888). Fails both ways described above. Full traceback available on request.
Fix
Opened # — changes TreeNode.node_id/parent_id/children_ids, TreeKnowledge.root_node_id/nodes, BaseKnowledge.id, and PaperKnowledgeCard.paper_id from UUID to str (accepts both UUIDs and readable slugs), and has paper_flow pass AgentOutputSchema(out_type, strict_json_schema=False) instead of a bare type. Verified: the unmodified README example now completes end-to-end with zero validation errors against the same real paper, and the full test suite passes (232/233 — the one failure is pre-existing and unrelated, a Windows path-separator assertion in a local-file test).
Environment
openai-agents==0.18.1(resolved from the unpinned>=0.14floor)- Windows 11, Python 3.12.12
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with quantmind/flows/paper.py and quantmind/knowledge/_tree.py, then inspect the Paper and related knowledge models named in the issue. Run the README single-paper example against openai-agents 0.18.1 and the full test suite. Done means the example completes without schema or UUID validation errors and existing tests remain passing aside from the noted unrelated failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100