Avijit-Kumar-GIT / Avijit-Kumar-GIT/fella

folder-QA v2: deep multi-file / multi-hop tier (3+ files, derived join keys)

Open
#87 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
0
Forks
0
Avg merge
5h 19m
Merged PRs (30d)
72

Description

Sub-thread of #68, scoped out on its own since it's a distinct design problem from the other v2 tiers.

**What v1 covers today:** 2-file joins on a shared or near-shared column name, always with the join key spelled out or discoverable in one step (`shared_column_hints` from #58 usually names it directly). That's realistic for a first pass but it's the *easy* end of multi-file — real folders have longer chains.

**What's missing, additive to v1 (don't replace the easy/medium tier — real usage spans the whole range):**
- **3-5 file chains** — the answer requires walking through more than one join, not just A⋈B.
- **Derived join keys** — no shared column name to hint at; the model has to notice that `books.author` and `authors.author_name` (or an id vs. a display name) refer to the same entity before it can join, or match on a normalized/partial string rather than an exact key.
- **A figure reconciled across 2+ tables *and* a document** at once (today's `mf-doc-join` cases are one table + one document; push to two tables + a document).
- **The right files aren't named in the question** and aren't obviously the only plausible candidates — v1's "cluttered folder" cases use unrelated distractors; this tier needs *plausible-but-wrong* files (e.g. two tables that could both answer "spending" but only one is the right scope).

**Why this matters for the loop, not just the battery:** this is exactly the tier that would stress #58 (join hints) and give #79 (flag a question answered from only one table when a join was implied) real signal to fire on it's currently undertested because v1's joins are too easy to miss.

Deterministic gold is still achievable here (`gen.py` can compute it, it's just more files) — this tier doesn't need new grading infrastructure, unlike the research-questions tier.

Contributor guide

Open the contributing guide

Research direction

Start with gen.py and the existing v1 multi-file and mf-doc-join cases referenced in the issue. Trace how deterministic gold answers are generated and how join hints from #58 and single-table detection from #79 are exercised. Done means adding a distinct 3–5-file tier with derived keys, plausible distractors, and table-plus-document reconciliation without replacing existing tiers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sql
Domain
ai, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.