Runner.bench_func can't run different module's function?

I'm trying to run Runner.bench_func inside a package like this:

└── format
    ├── benchmarks
    │   ├── bm_format.py
    │   ├── bm_fstring.py
    │   ├── bm_percentage.py
    │   └── __init__.py
    ├── __init__.py
    └── run.py

Inside format/run.py the code will run benchmarks' function like this:

import perf
from .benchmarks import BENCHS


def run():
    runner = perf.Runner()
    for bm_name, bm_func in BENCHS:
        runner.bench_func(bm_name, bm_func)


if __name__ == '__main__':
    run()

When running this inside cli, it will raise RuntimeError and ModuleNotFoundError:

➜  python-string-format python -m format.run
Traceback (most recent call last):
  File "/tmp/python-string-format/format/run.py", line 2, in <module>
    from .benchmarks import BENCHS
ModuleNotFoundError: No module named '__main__.benchmarks'; '__main__' is not a package
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tmp/python-string-format/format/run.py", line 12, in <module>
    run()
  File "/tmp/python-string-format/format/run.py", line 8, in run
    runner.bench_func(bm_name, bm_func)
  File "/usr/lib/python3.6/site-packages/perf/_runner.py", line 594, in bench_func
    return self._main(name, sample_func, inner_loops, metadata)
  File "/usr/lib/python3.6/site-packages/perf/_runner.py", line 508, in _main
    bench = self._master()
  File "/usr/lib/python3.6/site-packages/perf/_runner.py", line 768, in _master
    bench = self._spawn_workers()
  File "/usr/lib/python3.6/site-packages/perf/_runner.py", line 724, in _spawn_workers
    suite = self._spawn_worker(calibrate)
  File "/usr/lib/python3.6/site-packages/perf/_runner.py", line 665, in _spawn_worker
    % (cmd[0], exitcode))
RuntimeError: /usr/bin/python failed with exit code 1

Do I did wrong something, or the Runner.bench_func can only run inside its module?

Thanks.

Reproduce the report with the package layout and command python -m format.run, starting from format/run.py and Runner.bench_func. Then inspect the worker-spawning path in perf/_runner.py, especially the traceback's _spawn_workers and _spawn_worker entries. Done means the benchmark function from another package module runs without the reported ModuleNotFoundError and RuntimeError.