Triple dispatching our way into a smaller interpreter

Feature or enhancement

Proposal:

This is orthogonal to https://github.com/python/cpython/issues/148506. However, the problem remains the same. I'll repeat it again here: we have hit the limit of the computed goto/switch-case interpreter size such that we are seeing compiler bugs in all three: MSVC, Clang, and GCC. (I just fixed another compiler bug in Clang 22 due to interpreter size last week). The tail calling interpreter is the long-term solution. In the short term, we need another one. It is imperative that we reduce the size of the interpreter.

Instrumentation takes up 21 opcodes. This presents an opportunity for significant interpreter size reductions and recovery of opcodes. With this, we can remove all instrumented instructions. The INSTRUMENTED_X instructions become pseudo instructions with oparg > 255 . We store the instrumented pseudo-instruction in the instrumentation tools in get_tools_for_instruction instead of the bytecode, so that we can exceed the 255 limit.

To achieve instrumentation, we just swap out the interpreter dispatch table. The key observation is to repurpose the current TRACE_RECORD instruction to a VARIABLE_DISPATCH operation and change that single instruction to use call threading. This call threading instruction will support both JIT and instrumentation modes. We also move all instrumentation functions to small helper functions automatically using the cases generator. This is a small change, as we already refactored the interpreter to almost support call-threading due to the tail calling interpreter work, where each opcode is implemented as a function.

// For instrumentation
static void *instrumented_dispatch_table[256] = {
    [FOR_ITER] = &&VARIABLE_DISPATCH,
}

static funcptr instrumented_targets_table[256] = {
    [FOR_ITER] = &_CALL_FOR_ITER,
    [INSTRUMENTED_FOR_ITER] = &_CALL_INSTRUMENTED_FOR_ITER,
}

// For JIT
static void *tracing_targets_table[256] = {
    [FOR_ITER] = &&VARIABLE_DISPATCH,
}

inst(VARIABLE_DISPATCH, (--)) {
    next_instr = this_instr;
    if (dispatch_table_var == instrumented_dispatch_table) {
        if (HAS_INSTRUMENTED_OPCODES[opcode]) {
            opcode = get_instrumented_opcode(inst);
            _CALL_ARGS = instrumented_targets_table[opcode](_CALL_ARGS);
            DISPATCH();
        }
        else {
            DISPATCH_NON_INSTRUMENTED();
         }
    }
    else {
        assert(dispatch_table_var == tracing_targets_table);
        // Do what _TRACE_RECORD currently does in the JIT.
    }
}

This will remove all instrumented opcodes from the main interpreter. Runtime instrumentation might be slower, but still faster than the old days of sys.settrace(). However instrumentation pauses will be significantly lower, as we won't have to scan bytecode to instrument, instead we just check the current tools and swap the dispatch table. Finally, the base interpreter should be faster too.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Start by reading the interpreter dispatch logic around TRACE_RECORD, VARIABLE_DISPATCH, get_tools_for_instruction, and the cases generator mentioned in the proposal. Trace how instrumentation and JIT dispatch tables currently work. Done means removing instrumented instructions from the main interpreter while preserving instrumentation and JIT behavior and reducing interpreter size.