python / python/cpython

Extend `FOR_ITER` specializations to cover other indexable sequences

Open
#145,668 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3.15 interpreter-core performance
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Currently, we specialize FOR_ITER for both tuple and list. We also handle tuple and lists specially in GET_ITER to support this specialization.

We should extend this specialization of iteration over sequences, using virtual iterators to bytes, str, bytesarray and others.
To do this, we should:

  • Add a tp_getiteritem slots to PyTypeObject for sequences that can be efficiently indexed and cannot raise except for out-of-bounds access.
  • Add a specialization of FOR_ITER for these classes.
  • Specialize GET_ITER for:
    • iterators
    • indexable sequences (those classes with a tp_getiteritem slot)
  • Specialize SEND for indexable sequences which should also improve JIT coverage, as unspecialized _SEND is currently a major cause of jit optimization failures.

Combined with https://github.com/python/cpython/issues/145667 this will allow effective specialization of a wider range of iterables and allow that specialization not only for for loops, but also for yield from loops.
It will add 3 instructions (4 new, 1 removed) which a reasonable

Linked PRs
  • gh-145670
  • gh-147967

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing FOR_ITER and GET_ITER specializations, then inspect PyTypeObject and the SEND specialization. The work is complete when indexable sequences such as bytes and str use virtual iterators across for loops and yield from loops, with the requested new instruction coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.