tarantool / tarantool/crud

select/pairs execution plan

Open
#348 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

8sp customer feature
Dominant language
Lua
Stars
43
Forks
17
Avg merge
28m
Merged PRs (30d)
2

Description

From time to time, application developers ask: "What's the cost of this request?" And the answer is not always clear even for a crud developer.

select/pairs could be used for requests more complicated than just a primary key get, especially for paginations. Will this request trigger map reduce? Will it trigger tuple fullscan on scroll? Due to implementation, sometimes the order of conditions may change the result. Based on this info, application developer may realize that they need a different index or sharding architecture. Some of this info already presents in metrics, but could be hard to get a grip of what exactly is going on with a single request from the aggregated info.

Some planning info already presents on router: see plan.lua. It's not exposed anywhere yet. It contains the info about which storage or storages would be used and which index would be used to select. We may extend it with sharding key info. Some planning info, like tuple scroll, could be extracted only from storage. We don't have an explicit plan table there yet, but it is possible to built in without any drastic additional cost.

The proposal is to allow a user call crud.select/crud.pairs with {dry_run = true}/{plan = true} (or maybe with new handle select_plan) and return a table with exhaustive info about what this request will do. Since this option would be used only for debug, one of the approaches that could be used to build a plan is to actually execute the request and write down the useful info from the storage-site, but it seems preferable to implement an actual dry run (so user may check potentially dangerous requests too).

We shouldn't forget to skip such requests in metrics monitoring info (or decide that we will track then, but the question should be raised either way).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading compare/plan.lua and select/executor.lua, then trace the crud.select and crud.pairs entry points and their metrics handling. Determine what plan data is available at the router and storage layers, and how a dry-run or plan option should expose it without misleading metrics. Done means the proposal has a settled interface, exhaustive plan contents, and tests covering the selected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
backend-api-design, databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.