Make timeit CLI function available from python

This is a summary of the discussion and conclusion from the discourse thread, as suggested by @terryjreedy, but you can read the entire discussion there if you want more info :)

Feature or enhancement

Proposal:

Currently, timeit's Python interface offers timeit.timeit and timeit.repeat, and timeit.autorange. timeit's command-line interface provides a lot of features over these functions, which are meant for interactive use.

These include:

• Printing and formatting to human-readable results, including unit conversion
• Verbose mode
• Automatically determining a number, if not provided, using autorange.
• Warning if the tests are unreliable

It makes sense for these features to not be included in most of timeit's Python interface, since those are APIs that aren't only meant for interactive use. Printing, formatting, and warning aren't always needed when you can just get the return value and calculate what you need.

However, there is a use case for these interactive features to be available from Python as-well. Many quick tests are easier to do from the REPL or a proper IDE where you have variables and statement editing, rather than just passing the code as a plain string. This is especially true when some of the arguments can be multi-line python code.

There should be a way for users to get the human-friendly features when using timeit from the Python, and not just from the CLI.

timeit.run

The proposed way to do this is by splitting timeit.main to roughly 2 parts: argument parsing, and actually running the tests. The former will stay in timeit.main, and the latter will move to a new timeit.run function which will be suited for interactive use from Python.

I have opened a draft PR for this here, and will continue working on it in the following days: #119165

Example usage (draft)

import random
import time
import timeit

def test_random():
     if random.random() > 0.9:
             time.sleep(1)

timeit.run(test_random, verbose=True, number=0, time_unit="nsec")
# 1 loop -> 1.84e-05 secs
# 2 loops -> 2.09e-05 secs
# 5 loops -> 1.001 secs
#
# raw times: 1e+09 nsec, 5.04e+04 nsec, 1e+09 nsec, 2.001e+09 nsec, 3.69e+04 nsec
# 
# 5 loops, best of 5: 7380 nsec per loop
# :0: UserWarning: The test results are likely unreliable. The worst time (4.001e+08 nsec) was more than four times slower than # # the best time (7380 nsec).

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/timeit-make-cli-function-available-from-python/53339

Linked PRs

• gh-119165

Start by reading the timeit.main entry point and the proposal for separating argument parsing from test execution. Review draft PR #119165 to understand the work already underway; done means providing the proposed interactive Python API while preserving the command-line behavior and features described here.