Ensuring good specialization while making progress in jitted code

The problem

For obvious reasons, it is necessary that jitted code makes progress: it can't spin in an infinite loop making no progress.
To ensure this happens, we require that any executor attached to an ENTER_EXECUTOR instruction executes at least one instruction (opcode, not uop) before returning to the interpeter.
This means that the first instruction in a trace cannot exit, which prevents specialization.
Worse, some unspecialized instructions, notably CALL and SEND cannot be jitted at all, due to complex control flow.

Dropping the requirement for progress

We could drop the requirement for progress in exceutors and have ENTER_EXECUTOR handle it: if it appears that an executor has no made progress when returning to ENTER_EXECUTOR then do not dispatch normally to the next ENTER_EXECUTOR but dispatch to the underlying executor. @brandtbucher likes this approach.

There are a few issue with this approach, however:

• It makes attaching traces to side exits more complicated, as we need to check that we aren't attaching a no-progress trace to a no-progress exit
• It may prevent, or at least complicate, tail-calling from jitted code into the interpreter in future, as we will need to communicate that progress has been made
• It complicates ENTER_EXECUTOR

A solution

Instead of worrying about what happens when a trace starts with an instruction that needs to be specialized, or cannot be jitted, we can make sure that that situation never happens or, at least, happens very rarely.

There are three possible ways to start tracing, plus a fourth that we may add in the future

1. At a backwards jump
2. (In the future) at the start of a function
3. After a guard failure; a side exit.
4. At the end of another executor

Cases 1 and 2 are trivial, as they start with a non-specializable instruction, so progress is guaranteed.
If an executor makes progress, then we don't to make progress on side traces, so case 3 is not a problem either.
It is only case 4 that is causing trouble.

If we can only end traces at locations that aren't problematic, then we are guaranteed progress.

Traces can end if we either run out of resources (trace too long, or too deep) or we hit an instruction that we can't handle in the JIT.
We can deal with the too long case by backing up to a known good ending point.
We can't do anything about hitting an instruction that can't be jitted, but we can reduce the number of such instructions so that we can jit through them and not end on specializable instructions.

Instructions that cannot be handled in the JIT

Many of these are monitoring and other rare instructions that we don't much care about.
However there are the specializable instructions (families) that we do care about.
From (old) stats it appears that CALL and SEND form the bulk of the cases where we stop tracing on a specializable instruction.

Tasks

• Complete specialization for CALL, including adding "generic specializations"
• Complete specialization for SEND, including adding "generic specializations"
• Track last known good exit point during tracing/optimization and truncate overly long traces to that point

Specializations for CALL

1. Make sure that if more specific specializations for Python functions fail, we still specialize to CALL_PY_GENERAL
2. Add CALL___CALL___PY for __call__ implemented in Python, and CALL___CALL___C for __call__ implemented in C as the "generic specializations". We should still guard on the type, even though they are "generic".

Specializations for SEND

The SEND instruction is used in yield from and await.
We already specialize for generators and coroutines in yield from and await respectively, but
sequences like list and tuple are also common in yield from.
Async generators are also common, but that might just be our benchmarks.

It might be worth investigating the possibility of merging FOR_ITER and SEND to take advantage of the
existing specializations for sequences.

As a first step, however, we should add a non-generator, non-coroutine, "generic specialization" so that the JIT
can handle it.

Inizia separando le tre attività elencate: specializzazioni generiche per CALL, specializzazione generica per SEND e tracciamento di un ultimo punto di uscita noto come valido durante il tracing o l’ottimizzazione. Leggi prima il comportamento relativo a ENTER_EXECUTOR, CALL, SEND e FOR_ITER. Il lavoro è completato quando i casi CALL e SEND indicati possono essere gestiti dal JIT e le trace eccessivamente lunghe vengono troncate fino a un punto finale noto come valido.