Proposal: individualized context for distinct modules
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
Currently, all modules share a same context named `sessionctx.Context`, and it brings some problems such as cycle dependency and unnecessary coupling. It's better to have individualized context for distinct modules. With this change, we can improve the maintainability of the codebase with the advantages below:
- Each context is only responsible for its own module, and we do not need to have a context with logics from all modules. So it is easier to understand the code.
- We can reduce the coupling between modules and avoid cycle dependencies.
- We can easily add new fields to the context of a module without affecting other modules.
For example, we can define a new interface `EvalContext` to provide information for expression evaluations:
```go
type EvalContext interface {
BaseEvalInfo() BaseEvalInfo
}
```
And then we can define a new context `evalContext` to implement it
```go
// evalContext implements EvalContext
type evalContext struct {
sctx sessionctx.Context
}
func (ctx evalContext) BaseEvalInfo() BaseEvalInfo {
return BaseEvalInfo{
TypeCtx: ctx.sctx.GetSessionVars().StmtCtx.TypeCtx(),
ErrCtx: ctx.sctx.GetSessionVars().StmtCtx.ErrCtx(),
SQLMode: ctx.sctx.GetSessionVars().SQLMode,
}
}
```
Different with the previous design, the above example just uses `sessionctx.Context` to provide some plain "state" instead of providing "semantics." `EvalContext` which is a wrapper of `sessionctx.Context` is the one that provides "semantics" for expression evaluations. In this way, different modules can depend on the packages they really need, and we can avoid cycle dependencies. The design of distinct contexts also brings flexibility because each module can have their own assumptions about the context without putting them together into a single context.
## Some New Contexts to Introduce
Some important new contexts are listed below:
### EvalContext
As we mentioned above, `EvalContext` provides information for expression evaluations:
```go
type EvalContext interface {
BaseEvalInfo() BaseEvalInfo
Level() EvalContextLevel
FreezeToLevel(level EvalContextLevel) (EvalContext, error)
}
```
Only `BaseEvalInfo` is required for now to get the basic information for evaluating expressions. BaseEvalInfo is defined as a immutable struct:
```go
type BaseEvalInfo struct {
typeCtx types.Context
errCtx erctx.Context
sqlMode mysql.SQLMode
...
}
```
For most expression evaluations, `BaseEvalInfo` is enough. However, for some other special functions, extra information is required. For example, `tidb_is_ddl_owner` needs to know whether the current TiDB node is a DDL owner that is not included in `BaseEvalInfo`, so in this case the input `EvalContext` should be cast to another interface:
```go
type ExtenedEvalContext struct {
EvalContext
IsDDLOwner() bool
...
}
```
Some extra information can be added to `ExtenedEvalContext`.
Providing multiple interfaces for eval context is that not cases need all information in an evaluation process. For example, if we want to evaluate an expression from a generated column, `BaseEvalInfo` is enough. Make an easy way to build `EvalContext` is helpful for some scenes that we cannot get a `sessionctx.Context` directly, such as the scene of lighting import.
Since we have multiple interfaces for `EvalContext`, the method `Level` returns which level of information the current context can provide. It's return value is `EvalContextLevel` and is defined as below:
```go
type EvalContextLevel int
const (
EvalContextLevelBase EvalContextLevel = iota
EvalContextLevelExtened
EvalContextLevelFull
)
```
- `EvalContextLevelBase` means the context only provides `BaseEvalInfo`. It is simple and efficient to build and copy. It is always used in some simple cases, such as evaluating an expression from a generated column.
- `EvalContextLevelBase` means the context provides some more extended info. This type of context is not easy to build, it always reads the information from a inner `sessionctx.Context`. But we can copy it from to a new one to detach it from the inner `sessionctx.Context`.
- `EvalContextLevelFull` provides the full context, all expressions can use it as the input. However, it cannot be detached from the inner `sessionctx.Context` because it should provide some complex states that is hard to copy.
`Freeze` is a special method that returns a new `EvalContext` with the specified level. A "frozen" context is not only thread safe, it's inner state should be copied from the previous context and can keep immutable even if the inner `sessionctx.Context` is changed. This is useful for some cases, such as cursor fetch mode. A context with a higher level can be frozen to a lower level, but a context with a lower level cannot be frozen to a higher level. For example, a context with level `EvalContextLevelFull` can be frozen to `EvalContextLevelExtened` or `EvalContextLevelBase`, but a context with level `EvalContextLevelBase` cannot be frozen to `EvalContextLevelExtened`. Freeze a context with any level to `EvalContextLevelFull` is also not allowed, because this level is not able to be detached from the inner `sessionctx.Context`.
We should also add a new method `EvalContextRequirement` to `expression.Expression` to indicate which level of `EvalContext` is required for the expression evaluation:
```go
type Expression interface {
...
EvalContextRequirement() EvalContextLevel
Eval(ctx EvalContext, row chunk.Row) (types.Datum, error)
...
}
```
`EvalContextRequirement` is useful in some scenes such as cursor fetches. If a statement includes an expression which requires a eval context with level `EvalContextLevelBase` or `EvalContextLevelExtened`, that means we can fetch the data lazily in cursor fetch mode by cloning the context ignoring whether another new statement is created. Otherwise, we should fall back to
the previous implement to fetch all data before returning the cursor.
### ExprBuildContext
`ExprBuildContext` is a context for building expressions:
```go
type ExprBuildContext interface {
EvalContext
// some other methods
...
}
```
In practice, the procedure of building an expression always needs to do some evaluations, so it also extends the interface `EvalContext`.
Some methods will replace `ExprBuildContext` with `sessionctx.Context` as the input argument:
```diff
type functionClass interface {
// getFunction gets a function signature by the types and the counts of given arguments.
- getFunction(ctx sessionctx.Context, args []Expression) (builtinFunc, error)
+ getFunction(ctx ExprBuildContext, args []Expression) (builtinFunc, error)
// verifyArgsByCount verifies the count of parameters.
verifyArgsByCount(l int) error
}
// NewFunction creates a new scalar function or constant via a constant folding.
- func NewFunction(ctx sessionctx.Context, funcName string, retType *types.FieldType, args ...Expression) (Expression, error) {
+ func NewFunction(ctx ExprBuildContext, funcName string, retType *types.FieldType, args ...Expression) (Expression, error) {
return newFunctionImpl(ctx, 1, funcName, retType, defaultScalarFunctionCheck, args...)
}
```
### PlanContext
`PlanContext` is a context for building and optimizing plans:
```go
type PlanContext interface {
EvalContext
// some other methods
...
}
```
`PlanContext` also extends `EvalContext` because it needs to do some evaluations in building and optimizing procedure.
Some methods will replace `PlanContext` with `sessionctx.Context` as the input argument or return value:
```diff
type Plan interface {
...
- SCtx() sessionctx.Context
+ Ctx() PlanContext
...
}
```
```diff
- func Optimize(ctx context.Context, sctx sessionctx.Context, node ast.Node, is infoschema.InfoSchema) (plan core.Plan, slice types.NameSlice, retErr error) {
+ func Optimize(ctx context.Context, pctx core.PlanContext, node ast.Node, is infoschema.InfoSchema) (plan core.Plan, slice types.NameSlice, retErr error) {
...
}
```
### table.MutateContext & table.AllocatorContext
These two contexts are used to provide information for methods in interface `tables.Table`.
`table.MutateContext` is defined as:
```
type MutateContext interface {
expression.BuildContext
// ... some other methods
}
```
It is used to provide context when we are mutating a table and should extend `expression.BuildContext` to build expressions defined in table.
`AllocatorContext` is defined as:
```
type AllocatorContext interface {
// GetTemporaryTable returns some runtime information for temporary tables.
GetTemporaryTable(tbl *model.TableInfo) tableutil.TempTable
}
```
It only contains one method `GetTemporaryTable` to provide some runtime information for the temporary table to allocate id in session level instead of global level.
### ExecutorContext
`ExecutorContext` is a context for executor executing:
```go
type ExecutorContext interface {
EvalContext
// some other methods
...
}
```
`ExecutorContext` also extends `EvalContext` because it needs to do some evaluations in the executing procedure. The type of `ctx` inner `BaseExecutor` will be replaced as `ExecutorContext`:
```diff
type BaseExecutor struct {
- ctx sessionctx.Context
+ ctx ExecutorContext
}
```
## Tasks
### Expression
- [x] #49415
- [x] #50661
- [x] #50691
- [x] #50861
- [x] #51070
- [x] #51149
- [x] #51171
- [x] #51223
- [x] #51236
- [x] #51259
- [x] #51298
- [x] #51381
- [x] #51419
- [x] #51477
- [x] #51563
- [ ] #51656
- [x] #51890
Contributor guide
Assessment
This issue has not been assessed yet.