Ensuring good specialization while making progress in jitted code

The problem

For obvious reasons, it is necessary that jitted code makes progress: it can't spin in an infinite loop making no progress.
To ensure this happens, we require that any executor attached to an ENTER_EXECUTOR instruction executes at least one instruction (opcode, not uop) before returning to the interpeter.
This means that the first instruction in a trace cannot exit, which prevents specialization.
Worse, some unspecialized instructions, notably CALL and SEND cannot be jitted at all, due to complex control flow.

Dropping the requirement for progress

We could drop the requirement for progress in exceutors and have ENTER_EXECUTOR handle it: if it appears that an executor has no made progress when returning to ENTER_EXECUTOR then do not dispatch normally to the next ENTER_EXECUTOR but dispatch to the underlying executor. @brandtbucher likes this approach.

There are a few issue with this approach, however:

• It makes attaching traces to side exits more complicated, as we need to check that we aren't attaching a no-progress trace to a no-progress exit
• It may prevent, or at least complicate, tail-calling from jitted code into the interpreter in future, as we will need to communicate that progress has been made
• It complicates ENTER_EXECUTOR

A solution

Instead of worrying about what happens when a trace starts with an instruction that needs to be specialized, or cannot be jitted, we can make sure that that situation never happens or, at least, happens very rarely.

There are three possible ways to start tracing, plus a fourth that we may add in the future

1. At a backwards jump
2. (In the future) at the start of a function
3. After a guard failure; a side exit.
4. At the end of another executor

Cases 1 and 2 are trivial, as they start with a non-specializable instruction, so progress is guaranteed.
If an executor makes progress, then we don't to make progress on side traces, so case 3 is not a problem either.
It is only case 4 that is causing trouble.

If we can only end traces at locations that aren't problematic, then we are guaranteed progress.

Traces can end if we either run out of resources (trace too long, or too deep) or we hit an instruction that we can't handle in the JIT.
We can deal with the too long case by backing up to a known good ending point.
We can't do anything about hitting an instruction that can't be jitted, but we can reduce the number of such instructions so that we can jit through them and not end on specializable instructions.

Instructions that cannot be handled in the JIT

Many of these are monitoring and other rare instructions that we don't much care about.
However there are the specializable instructions (families) that we do care about.
From (old) stats it appears that CALL and SEND form the bulk of the cases where we stop tracing on a specializable instruction.

Tasks

• Complete specialization for CALL, including adding "generic specializations"
• Complete specialization for SEND, including adding "generic specializations"
• Track last known good exit point during tracing/optimization and truncate overly long traces to that point

Specializations for CALL

1. Make sure that if more specific specializations for Python functions fail, we still specialize to CALL_PY_GENERAL
2. Add CALL___CALL___PY for __call__ implemented in Python, and CALL___CALL___C for __call__ implemented in C as the "generic specializations". We should still guard on the type, even though they are "generic".

Specializations for SEND

The SEND instruction is used in yield from and await.
We already specialize for generators and coroutines in yield from and await respectively, but
sequences like list and tuple are also common in yield from.
Async generators are also common, but that might just be our benchmarks.

It might be worth investigating the possibility of merging FOR_ITER and SEND to take advantage of the
existing specializations for sequences.

As a first step, however, we should add a non-generator, non-coroutine, "generic specialization" so that the JIT
can handle it.

Bắt đầu bằng cách tách ba tác vụ được liệt kê: các chuyên biệt hóa generic cho CALL, chuyên biệt hóa generic cho SEND và theo dõi điểm thoát hợp lệ được biết gần đây nhất trong quá trình tracing hoặc tối ưu hóa. Trước tiên, hãy đọc hành vi xung quanh ENTER_EXECUTOR, CALL, SEND và FOR_ITER. Hoàn tất có nghĩa là các trường hợp CALL và SEND đã nêu có thể được JIT xử lý, đồng thời các trace quá dài được cắt ngắn đến một điểm kết thúc hợp lệ đã biết.