Avijit-Kumar-GIT / Avijit-Kumar-GIT/fella
folder-QA v2: deep multi-file / multi-hop tier (3+ files, derived join keys)
- Dominant language
- Rust
- Stars
- 0
- Forks
- 0
- Avg merge
- 5h 19m
- Merged PRs (30d)
- 72
Description
Sub-thread of #68, scoped out on its own since it's a distinct design problem from the other v2 tiers.
**What v1 covers today:** 2-file joins on a shared or near-shared column name, always with the join key spelled out or discoverable in one step (`shared_column_hints` from #58 usually names it directly). That's realistic for a first pass but it's the *easy* end of multi-file — real folders have longer chains.
**What's missing, additive to v1 (don't replace the easy/medium tier — real usage spans the whole range):**
- **3-5 file chains** — the answer requires walking through more than one join, not just A⋈B.
- **Derived join keys** — no shared column name to hint at; the model has to notice that `books.author` and `authors.author_name` (or an id vs. a display name) refer to the same entity before it can join, or match on a normalized/partial string rather than an exact key.
- **A figure reconciled across 2+ tables *and* a document** at once (today's `mf-doc-join` cases are one table + one document; push to two tables + a document).
- **The right files aren't named in the question** and aren't obviously the only plausible candidates — v1's "cluttered folder" cases use unrelated distractors; this tier needs *plausible-but-wrong* files (e.g. two tables that could both answer "spending" but only one is the right scope).
**Why this matters for the loop, not just the battery:** this is exactly the tier that would stress #58 (join hints) and give #79 (flag a question answered from only one table when a join was implied) real signal to fire on it's currently undertested because v1's joins are too easy to miss.
Deterministic gold is still achievable here (`gen.py` can compute it, it's just more files) — this tier doesn't need new grading infrastructure, unlike the research-questions tier.
Contributor guide
Research direction
Start with gen.py and the existing v1 multi-file and mf-doc-join cases referenced in the issue. Trace how deterministic gold answers are generated and how join hints from #58 and single-table detection from #79 are exercised. Done means adding a distinct 3–5-file tier with derived keys, plausible distractors, and table-plus-document reconciliation without replacing existing tiers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- ai, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100