Instagram / Instagram/LibCST

MatcherDecoratableTransformer >3x slower than CSTTransformer, despite using no decorators

Open
#880 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.9k
Forks
229
PR merge metrics
No merged PRs in 30d

Description

```python
>>> flake8_trio.visitors.flake8triovisitor.Flake8TrioVisitor_cst.__bases__ = (libcst.CSTTransformer, abc.ABC)

>>> timeit.timeit(tests.test_flake8_trio.test_910_permutations, number=5)
52.892802361995564
>>> flake8_trio.visitors.flake8triovisitor.Flake8TrioVisitor_cst.__bases__ = (libcst.matchers.MatcherDecoratableTransformer, abc.ABC)

>>> timeit.timeit(tests.test_flake8_trio.test_910_permutations, number=5)
192.94241196200164
```
We have a [meaty testcase](https://github.com/Zac-HD/flake8-trio/blob/main/tests/test_flake8_trio.py#L426) in [flake8-trio](https://github.com/Zac-HD/flake8-trio) that runs through ~40k source code permutations and runs a [visitor](https://github.com/Zac-HD/flake8-trio/blob/main/flake8_trio/visitors/visitor91x.py) on it (the visitor doesn't do any source code transforming atm, but will do in the future). It's understandable that it had a significant slowdown when switching from ast to libCST, and that using decorators would also add some overhead, but I was very surprised when `MatcherDecoratableTransformer` introduces a ~3.6x slowdown on it's own over `CSTTransformer`!
That made me rewrite the code not to use any decorators, but if I try changing the base visitor class to inherit from `MatcherDecoratableTransformer` it slows down significantly despite not using a single matcher decorator anywhere.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.