describe_table summarizes a five-row sample with the shape of a whole-table profile
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 44
- Forks
- 1
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 142
Description
describe_table samples five rows (SAMPLE_ROWS in pkg-py/src/commons/_tools.py, n_sample in pkg-r/R/data-source.R) and then summarizes them with ellmer::df_schema() (and something else on the Python side). We shouldn't be computing those summaries on the sample. Currently, we get around this by telling the model that the summaries are being computed on a subset of the data and not to read into them too much, but we really shouldn't be computing them in the first place. At the same time, it's nice if we can describe the table enough that we can write code against it. What does the model have access to already? If that's enough, should we just remove that line (and the prompting associated with it)?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with SAMPLE_ROWS in pkg-py/src/commons/_tools.py and n_sample in pkg-r/R/data-source.R, then trace the summary generation and related prompting in both implementations. Check what table information the model already receives and whether it is sufficient for writing code. Done means sampled rows no longer produce whole-table summaries while the table remains describable enough for code generation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100