Set an upper limit on total facet suggestion time for a page
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 11.5k
- Forks
- 904
- Avg merge
- 4d 17h
- Merged PRs (30d)
- 18
Description
If a table has 100 columns the facet suggestion code will currently run 100 times, taking a max of facet_suggest_time_limit_ms which defaults to 50ms per column:
So for 100 columns, that's 100 * 50ms = 5s total time that might be spent attempting to calculate facets on a large table!
I should implement a hard upper limit on the total amount of time taken suggesting facets - probably of around 500ms. If it takes longer than that the remaining columns will not be considered.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in datasette/facets.py at lines 142-162, where facet suggestions are calculated for each column. Trace how the per-column facet_suggest_time_limit_ms is applied, then add a total page-level limit of about 500ms so remaining columns are skipped once it is exceeded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100